An always-on companion for your Mac, built on GitHub Copilot. It sees what you point at, answers out loud and draws the explanation right on your screen.
How it works
Press one shortcut and Ink Brain is there, over whatever you are reading, watching or building.
Press ⌥⇧Space, then circle, underline or click a figure, an equation or a photo.
Speak or type, in your own words. Keep talking for follow-ups, or go hands-free and start with “Ink Brain”.
It talks you through it with sketches, motion graphics and 3D models, built up step by step on your screen.
Your screen and your marks→a GitHub Copilot session→live tools draw on a transparent overlay
See it explained
Ink Brain decides how to teach. It sketches on the page for diagrams, builds motion graphics for maths and dense text, and models physical things in 3D, narrating each step as it draws.
Numbered notes, arrows and highlights drawn onto the figure you circled, one idea at a time.
Boards where equations rearrange themselves and graphs animate their variables, with real numbers.
Models you can turn around, with parts that move on their real hinges, come apart or are cut away.
Motion graphics
Circle a formula and Ink Brain builds it up on a board in the style of 3Blue1Brown: circles that spin, graphs that trace themselves and equations that rearrange, narrated step by step.
3D models
Physical things become detailed models you can cut open, take apart and set in motion, one narrated step at a time. Drag a model to look around it.
None of these are videos. Each demo is written in the same scene and board language Ink Brain's model writes, and drawn live in your browser by Ink Brain's own engine.
The film
A research figure, a dense definition and a confusing pinout, each circled, asked about out loud and explained on a real screen.
More than an answer
Ink Brain listens, reads, remembers and shows you how, without ever leaving the thing you are looking at.
Replies are spoken in an on-device neural voice, and the mic stays open for about 12 seconds so follow-ups just work.
It reads the article, thread or docs behind what you circled, rather than answering from memory.
Pull an image out of the screen, re-render it, or lay a live 3D or animated version over the original.
Your projects, devices, preferences and past conversations stay on your Mac, and you can turn any of it off.
With your approval, it moves the mouse and types to demonstrate a step, and stops the moment you move the mouse.
Turn on Talk hands-free in the menu bar and start any question with “Ink Brain”.
Ask about a video or a call playing on your Mac. Ink Brain transcribes it on-device, so it knows what was just said.
Works with Copilot
Install the extension and every new Copilot CLI or Copilot app session gets Ink Brain's tools, so it can explain, sketch and speak right where you are working.
~/ink-brain $ npm run extension:install
Installed ~/.copilot/extensions/ink-brain
New Copilot CLI and desktop app sessions load it automatically; run /clear (or restart) in sessions that are already open.
~/my-project $ copilot
❯ Walk me through this retry loop on my screen
● live_look Looked at your screen
● live_explain “How the retries back off” · 4 steps
● live_speak Narrating on your MacPrivacy
Nothing is captured until you ask, and what Ink Brain remembers never leaves your Mac.
Each question sends one screenshot, plus a zoomed crop around your marks, to the Copilot session you chose. The overlay is excluded from screenshots.
Speech recognition and the neural voice both run on your Mac, unless you choose to allow Apple's server speech.
Memory lives in ~/.ink-brain/memory with private file permissions. Screen history skips password managers and is deleted after 30 days.
Ink Brain only moves the mouse after you approve the listed steps, and stops when you press Esc, press the shortcut or move the mouse.
Panels run model-written code in sandboxed frames with no network access, and the overlay blocks web requests.
Pages are read in a hidden, signed-out window that loads no images or media. Set it to ask before each new site.
Under the hood
The model describes what it wants to show as data, such as shapes, boards and scenes. The overlay validates it and renders it in seconds, so it rarely has to write code.
Ink Brain Electron
Your screenshot, your marks and your question go to a Copilot session with Ink Brain's live tools.
rough.js sketches, KaTeX motion graphics and three.js models, rendered in sandboxed panels.
On-device speech recognition in, a neural Kokoro voice out, with Apple voices filling in while it loads.
A small Swift helper captures the screen with ScreenCaptureKit, listens, speaks and drives input.
Download
Free and open source. One command checks your Mac, installs Ink Brain and adds it to your Applications folder.
curl -fsSL https://inkbrain.app/install.sh | bash
Paste it into Terminal. It installs into ~/.ink-brain/app, adds Ink Brain to your Applications folder and starts it. Run it again any time to update. Read the script
git clone https://github.com/A-Georgiou/ink-brain.git
cd ink-brain
npm install
npm start
Prefer to see every step? Clone it and run it yourself. npm run doctor checks the helper, permissions, screen capture and voices.
copilot loginFAQ
Yes. Ink Brain is open source under the MIT licence. It answers using your own GitHub Copilot subscription, through the Copilot SDK.
Any Mac on macOS 14 or later, with Apple silicon or Intel. Ink Brain is macOS only for now.
Claude Opus 5.5 by default, through Copilot. Set model in ~/.ink-brain/config.json to choose another, and Ink Brain falls back to another vision model if yours is unavailable.
Nothing until you ask. Each question sends one screenshot, plus a zoomed crop around your marks, to the Copilot session you chose. Speech recognition and the voice run on your Mac.
Yes. Set hotkey in ~/.ink-brain/config.json, or the INK_BRAIN_HOTKEY environment variable, then restart Ink Brain. Open settings file in the menu bar takes you straight there.
Run the installer again to update. To uninstall, run curl -fsSL https://inkbrain.app/install.sh | bash -s -- --uninstall. Your settings and memory stay in ~/.ink-brain until you delete it.
No. Ink Brain is an independent open source project built on the GitHub Copilot SDK.