[feat] expose tao::Window::set_cursor_position

[CaptainTux]

Describe the problem

I want to reposition the mouse pointer within the tauri window.
I have read that winit has functions to do that, so I would assume tao can also do that.
Since tao is not exposed by tauri, I cannot use those capabilities.

Describe the solution you'd like

I would like an API function where I can pass coordinates relative to the top left corner of the tauri window which would reposition the mouse pointer.
For example, if I would do set_pointer_position(0,0), the cursor would jump to the top left corner.

Alternatives considered

I have created a custom tauri command using winapi, which looks like this:

#[tauri::command]
fn set_pointer_position(window: tauri::Window, x: i32, y: i32) {
  let inner_size = window.inner_size().unwrap();
  let outer_size = window.outer_size().unwrap();
  let pos = window.inner_position().unwrap();
  unsafe {
    if winapi::um::winuser::SetPhysicalCursorPos(pos.x + x, pos.y + y + (outer_size.height as i32 - inner_size.height as i32)) == 0 {
      panic!("SetCursorPos failed");
    }
  }
}

Additional context

No response

[CaptainTux]

I have tried using the exposed API function.
unfortunately, it hasn't worked for me.
I am on arch linux, kernel 5.17.3-arch1-1 build, and
npm run tauri info yields this:

> my_project@0.1.1 tauri
> tauri "info"


Environment
  › OS: Arch Linux Unknown X64
  › Node.js: 17.9.0
  › npm: 8.7.0
  › pnpm: Not installed!
  › yarn: 1.22.18
  › rustup: 1.24.3
  › rustc: 1.60.0
  › cargo: 1.60.0
  › Rust toolchain: stable-x86_64-unknown-linux-gnu

Packages
  › @tauri-apps/cli [NPM]: 1.0.0-rc.7(outdated, latest: 1.0.0-rc.8)
  › @tauri-apps/api [NPM]: 1.0.0-rc.3(outdated, latest: 1.0.0-rc.3)
  › tauri [RUST]: git+https://github.com/tauri-apps/tauri#8e00f0904c9d7385a5fc79a32b3bb2397699c305 (1.0.0-rc.6),
  › tauri-build [RUST]: git+https://github.com/tauri-apps/tauri#8e00f0904c9d7385a5fc79a32b3bb2397699c305 (1.0.0-rc.5),
  › tao [RUST]: 0.8.2,
  › wry [RUST]: 0.15.0,

App
  › build-type: bundle
  › CSP: unset
  › distDir: ../build
  › devPath: http://localhost:3000/
  › framework: React

my tauri command looks like this:

#[tauri::command]
fn set_pointer_position(window: tauri::Window, x: i32, y: i32) {
    let result = window.set_cursor_position(tauri::Position::Physical(tauri::PhysicalPosition{x, y}));
    if result.is_err() {
        panic!("setting cursor failed");
    }
    println!("cursor set {} {}", x, y);
}

and the console output is as expected.

// relevant cargo.toml data
[build-dependencies]
tauri-build = { git = "https://github.com/tauri-apps/tauri", features = [] }

[dependencies]
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
tauri = { git = "https://github.com/tauri-apps/tauri", features = ["api-all"] }
winapi = { version = "0.3.9" }
mouse-rs = { version = "0.4" }

[features]
# by default Tauri runs in production mode
# when `tauri dev` runs it is executed with `cargo run --no-default-features` if `devPath` is an URL
default = [ "custom-protocol" ]
# this feature is used used for production builds where `devPath` points to the filesystem
# DO NOT remove this
custom-protocol = [ "tauri/custom-protocol" ]

what is the problem here?
I am running the app with cargo run --no-default-features as was recommended in the discord server after npm run start.
I haven't tried it on Windows yet, planning to do that this weekend.

[lucasfernog]

It is not implemented yet on Linux, see https://github.com/tauri-apps/tao/blob/edf6e766704a3f10d62673e6be6f168a1daa0b15/src/platform_impl/linux/window.rs#L565

I documented this in the function but you'd need to build the docs manually to see it.

[amrbashir]

@CaptainTux I plan to implement this for linux soon. In a week maybe.

[amrbashir]

I guess I lied 😁, here it is tauri-apps/tao#373 if you want to test it and give feedback.

[lucasfernog]

set_cursor_grab doesn't exist on Linux too, maybe we could just set the cursor icon to grab?

[lucasfernog]

You can try it now by adding this to your Cargo.toml:

[patch.crates-io]
tao = { git = "https://github.com/tauri-apps/tao" }

Don't forget to run cd src-tauri && cargo update.

[amrbashir]

set_cursor_grab doesn't exist on Linux too, maybe we could just set the cursor icon to grab?

Cursor grab means to restrict the cursor movement from escaping the window coordinates, not just the icon change.

I wanted to implement it too in the PR but that one might need some trial and error to match the behavior of other platforms or close to it so I delayed it for now.

Relevant gtk apis:

• https://docs.gtk.org/gdk3/func.pointer_grab.html
• https://gtk-rs.org/gtk3-rs/stable/latest/docs/gdk/struct.Seat.html#method.grab
• https://gtk-rs.org/gtk3-rs/stable/latest/docs/gdk/struct.Device.html#method.grab

[CaptainTux]

I guess I lied grin, here it is tauri-apps/tao#373 if you want to test it and give feedback.

works!
However, I don't think it's in window coordinates, right now I have to do this:

#[tauri::command]
fn set_cursor_position(window: tauri::Window, x: i32, y: i32) {
  let inner_pos = window.inner_position().unwrap();
  let inner_size = window.inner_size().unwrap();
  let outer_size = window.outer_size().unwrap();
  if window.set_cursor_position
    (Logical(LogicalPosition{x: (x + inner_pos.x) as f64,
                             y: (y + inner_pos.y + (outer_size.height as i32 - inner_size.height as i32)) as f64
    })).is_err() {
    panic!("setting cursor failed");
  }
}

set_cursor_grab doesn't exist on Linux too, maybe we could just set the cursor icon to grab?

Cursor grab means to restrict the cursor movement from escaping the window coordinates, not just the icon change.

I wanted to implement it too in the PR but that one might need some trial and error to match the behavior of other platforms or close to it so I delayed it for now.

tao does not compile at the moment because of a type mismatch in set_cursor_grab (window.rs, line 581).
I inserted an Ok(()) which fixed it.

[lucasfernog]

It's not in window coordinates, it's in overall monitor coordinates (I think) so you're not limited to the window.

[lucasfernog]

My bad for the compilation error, can't belive I didn't see it. Pushed the fix.

[CaptainTux]

It's not in window coordinates, it's in overall monitor coordinates (I think) so you're not limited to the window.

That is correct, but the tooltip - from rust-analyzer - (I assume also the docs) claims the repositioning is in window coordinates, which is incorrect

[lucasfernog]

Ahh you're right. We'll need to change that.

[lucasfernog]

Opened a PR to fix it: tauri-apps/tao#375 thanks for catching this!