How to Use a Mouse Button as the Transcription Hotkey in Handy on Linux

[Josephur]

How to Use a Mouse Button as the Transcription Hotkey in Handy on Linux

I recently started using Handy, an open source local speech-to-text application, and wanted a very simple workflow:

Click middle mouse button → speak → click middle mouse again → transcription appears where I was typing.

Handy can do this on Linux, but there is a small catch.

By default, Linux uses Handy's Tauri Global Shortcut backend, which is mainly intended for keyboard shortcuts.

Handy also includes a newer backend called Handy Keys, which can detect raw mouse buttons through Linux.

evdev

The problem is that Handy Keys needs permission to access:

/dev/input/event*

and:

/dev/uinput

Without those permissions, mouse-button detection may not work.

I tested this on CachyOS + KDE Plasma + Wayland, but the same basic approach should work on most modern Linux distributions using systemd and udev.


1. Enable Handy Keys

Open Handy and go to:

Settings → Advanced

Turn on:

Experimental Features

A new Experimental section should appear farther down the page.

Find:

Keyboard Implementation

Change:

Tauri Global Shortcut

to:

Handy Keys

Then go back to the General section and click to set your shortcut key, mouse buttons should now work!

If mouse buttons work after doing this, you are good to go!

If Handy still does not detect your middle mouse button, continue below.


2. Check Your Linux Input PermissionsD

Open a terminal and run:

ls -l /dev/uinput /dev/input/event* | head

Then run:

id

On many systems the input devices will look something like:

crw-rw---- root ...

Your normal desktop user may not have permission to read them.

Handy Keys reads Linux input devices directly through:

evdev

It does not rely on KDE, GNOME, X11, or Wayland global-hotkey APIs for this.

So if you were about to start digging through Plasma's global shortcut permission settings, that is probably not the issue.

3. Add a udev Rule for Handy

The cleaner approach is to use

uaccess

instead of permanently adding your user to the Linux

input

group.

Run:

sudo tee /etc/udev/rules.d/70-handy-input.rules >/dev/null <<'EOF'
KERNEL=="uinput", TAG+="uaccess"
SUBSYSTEM=="input", KERNEL=="event*", TAG+="uaccess"
EOF

Then load the uinput kernel module:

sudo modprobe uinput

Reload the udev rules:

sudo udevadm control --reload-rules

Retrigger the devices:

sudo udevadm trigger

Now fully quit Handy, including the tray process, and launch it again.


4. Verify That Your User Has Access

Run:

for f in /dev/input/event*; do
    [ -r "$f" ] || echo "NO READ: $f"
done

[ -r /dev/uinput ] || echo "NO READ: /dev/uinput"
[ -w /dev/uinput ] || echo "NO WRITE: /dev/uinput"

If everything is configured correctly, this command should ideally print nothing.

If you see something like:

NO READ: /dev/input/event7

then your user still does not have permission to access that input device.

5. Assign the Mouse Button in Handy

Go back to:

Settings → Advanced → Experimental

Make sure this is still selected:

Keyboard Implementation: Handy Keys

Then go to:

Settings → General → Transcribe Shortcut

Click the current shortcut field.

Now press your middle mouse button.

Handy should detect something equivalent to:

MouseMiddle

Handy Keys also supports extra mouse buttons such as:

MouseX1
MouseX2

A thumb button may actually be the cleaner option if you use Linux's normal middle-click paste feature.

Why This Works

Handy's newer Handy Keys backend reads input directly from:

/dev/input/event*

On Linux, the underlying library maps mouse events such as:

BTN_MIDDLE → MouseMiddle
BTN_SIDE   → MouseX1
BTN_EXTRA  → MouseX2

This works independently of whether your desktop is using:

• KDE Plasma
• GNOME
• Wayland
• X11

The important part is that Handy has permission to access the raw Linux input devices.

The /dev/uinput permission is also used for Handy Keys' shortcut-blocking functionality.


One Important Middle-Mouse Caveat

On Linux, Handy Keys can detect mouse-button hotkeys, but it does not block the original mouse event.

That matters because middle mouse traditionally performs Linux's primary selection paste.

So pressing middle mouse can potentially do both:

Toggle Handy transcription
and:
Perform the application's normal middle-click action

If this becomes annoying, using a spare thumb button such as MouseX1 or MouseX2 is probably the cleanest solution.

Another option is to remap middle mouse to an unused key such as F13, then assign F13 to Handy.


The Result

After setting this up, my workflow is:

Middle click
↓
Speak
↓
Middle click
↓
Handy transcribes locally
↓
Text is inserted into the active application

No cloud speech service is required.

For English dictation I am currently using:

Parakeet TDT 0.6B v2

It is English-only and has an excellent speed/accuracy balance.


Links

Handy:

GitHub - cjpais/Handy: A free, open source, and extensible speech-to-text application that works completely offline. · GitHub

Handy Keys:

GitHub - handy-computer/handy-keys: cross-platform global keyboard shortcuts library · GitHub


Quick TL;DR

If Handy on Linux will not detect your mouse button:

1. Enable Experimental Features.
2. Change Keyboard Implementation to Handy Keys.
3. Give your desktop session access to

   /dev/input/event*

   and

   /dev/uinput

   with a udev

   uaccess

   rule.
4. Restart Handy.
5. Set Transcribe Shortcut to MouseMiddle, MouseX1, or MouseX2.

That was all it took on CachyOS with KDE Plasma and Wayland for me to properly get Handy working with a mouse hotkey.