Touchy
The assistant that saves you the hassle
Touchy works with your apps and knows the world around you so you can stay in the moment.
Join the public betaThe assistant that saves you the hassle
Touchy works with your apps and knows the world around you so you can stay in the moment.
Join the public betaBut access to such a model will not, by itself, give everyone a useful personal assistant. A useful assistant must inhabit the same physical and digital context as its user—seeing what they see, hearing what they hear, and acting through the tools and systems available to them. It must also be far more reliable at understanding and preserving its user’s intentions than any agent today, because it must act autonomously to carry them out. These two problems define our work: building agent harnesses that connect seamlessly with the world around you, and making powerful AI more aligned and useful to its users.
In Silicon Valley, everyone is using AI constantly at work: to code, write emails, and manage companies. It’s gotten smart enough to get a lot done.
But it’s not on every surface yet. The AI assistants available to consumers are clunky, unhelpful, and built around retrieval. Most people use AI to search. The rest of life is still very analog–you have to open 10 different apps every day to keep up with everything. Even then, there are daily occurrences that you must call out to others for help with.
One problem is the form factor–it shouldn’t be a textbox. It should be something that can see, hear, reach out and touch the world around it. This is why we’re building Touchy–to reimagine the interface between humans and intelligence technologies.
The way to do this isn’t to make flashy, addiction-forming virtual drugs. We are in the greatest epidemic in human history. We have these screens in our pocket that we want to look at 24/7, perpetually designed to drive our attention away from the things that matter.
Instead, a good assistant should return the minimal amount of information to help its user, then go away. Asymmetrically, it has to access orders of magnitude more information from the physical and digital worlds to be actually helpful. Filtering the information and tailoring it to the user is hard. Current models are not good enough to solve this yet! To successfully do this means solving many magnum opus research problems in artificial intelligence, including:
There are few universal experiences that everyone wants. Some people prefer more detail and others prefer less. Some people are busy with events and others are looking for things to do. Within a language, there are many different dialects and phrases that only exist in certain people’s lexicon. Eric Horvitz had “continuing to learn by observing” as one of his principles of user interfaces in 1999.
Technically speaking, the future of continual learning is threefold: changing the weights/architecture of a machine learning model, changing the context the model has access to, and an agent changing its own harness on the fly–deciding to recursively keep calling itself in interesting graph structures. We believe all three of these viewpoints have a place in our research directions.
An assistant needs to get information about the physical world to be helpful. However, most processes in the physical world are slower than digital processes. Helping a user assemble furniture, operate an appliance they’ve never used before, cook a new recipe, or make sense of signs on road trips requires reasoning about images and video states continuously for many hours.
It also unlocks the ability to learn anything. For example, if you want to skateboard, your assistant can tell what went wrong on each fall.
An effective assistant can model a user’s theory of mind effectively. Understanding when users want technology to intervene and when they want it to get out of the way. Being selectively proactive and driving helpfulness over engagement. Creating valuable interaction, rather than meaningless clicks. Giving just one signal, then having the assistant work without disturbing the user. Estimating when the user is uncertain or stuck.
Effective theory of mind can also increase the quality of a response. For example, exhibiting sarcasm requires a simulation of your interlocutor’s mind–you must be able to predict that the other person will know you’re joking.
In our time, there are many incidents where AI plays to its current strengths–coding and persistence. An OpenClaw user asked its agent to book a gym class–it figured out how to cancel others’ class signups to get the user to the goal. Frontier AI labs have reported hacks of businesses that go beyond. A model even tried social engineering–targeting certain maintainers of a codebase to give it access.
Currently, certain band-aid measures exist like the Claude Code auto classifier. However, we need to have airtight safety guarantees before we can give it access to every surface.
Currently, on-device models offer low latency and the best privacy guarantees. Thus, expanding the capabilities of on-device models is extremely important for personal AI to become ubiquitous while keeping the power with the user.
Conversely, the very best models will always live off-device. Further, we imagine the near-term supply-chain constraints will make inference compute hardware limited in consumer products. Thus, we want to find solutions to delegate computation to the cloud while letting users stay in control of their data.
Hence, Touchy is making a unique bet in the consumer assistant space–applying research-focused principles to an obsession with UX will redefine how we let intelligence into our personal lives.
