macOSpilot

AI

desktop ai assistant

SHOW HN2023

Legend

macOSpilot

Open-source macOS AI copilot using vision and voice

github.com

#macos#openai vision#ai copilot#electron#voice assistant

430points
159comments
#70of 3,109 in 2023
Top 3%of 2023 by points

How HN took it

DebatedReal support and real criticism

21 positive, 22 neutral or asking, 2 mixed, 19 critical, out of 64 comments about the project. Thread score -0.05.

Idea: 10 for, 5 againstDesign: 1 for, 3 againstExecution: 0 for, 2 againstPricing: 1 for, 1 againstPerformance: 0 for, 2 against

The thread, comment by comment

Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.

positive neutral or mixed critical about another commenter or something else the maker

What HN said

What people liked

  • Useful for learning unfamiliar software, e.g. Ableton Live

    From 2 comments
    • Just used it with the digital audio workstation Ableton Live. It is amazing! Its tips were spot-on. I can see how much time it will save me when I'm working with a software or…
      thomashop on HN
    • So glad when I saw this, thanks for sharing this! It was exactly music production in Ableton was the spark that lit this idea in my head the other week. I tried to explain to a…
      ralfelfving on HN
  • Cool idea and useful concept

    From 3 comments

Objections

  • Sends screenshots to a third party cloud service, privacy concern

    From 2 comments
    • I mean it does send a screenshot of your screen off to a 3rd party, and that screenshot will most likely be used in future AI training sets. So... beware when you use it.
      pelorat on HN
    • Worth mentioning that if you are in a corporate environment, running a service that sends arbitrary desktop screenshots to a 3rd party cloud service is going to run afoul of…
      swiftcoder on HN
  • Open source label misleading because it uses proprietary OpenAI API

    From 3 comments
    • How is this open source when using a proprietary model. Misleading title
      jimsimmons on HN
    • > Open Source > off to OpenAI Vision Pick one
      fake-name on HN
    • The title is clickbait. We have come across too many applications that are merely wrappers around OpenAI API calls, yet they still claim to be open source.
      mwmercury on HN
  • Electron is heavy for this use case

    From 2 comments
    • e-e-e-electron... for this..
      ProfessorZoom on HN
    • >macOSpilot runs NodeJS/Electron Can we have native aps, please? I would like the battery life on my M1 Mac to stay good. :)
      DeathArrow on HN

Requests

  • Text input and text output instead of voice

    From 3 comments
    • You should add an option for streaming text as the response instead of TTS. And also maybe text in place of the voice command as well. I have been tire-kicking a similar kind of…
      jondwillis on HN
    • I’d love to see a version of this that uses text input/output instead of voice. I often have someone sleeping in the room with me and don’t want to speak.
      quinncom on HN
    • I have a tangential question: my dad is old. I would love to be able to have this feature, or any voice access to an LLM, available to him via an easy-to-press external button.…
      qup on HN
  • Support for local or self-hosted models

    From 3 comments
    • Nice project, any plans to make it work with local LLMs rather than "open"AI?
      smcleod on HN
    • I would love to have something like this but using an open source model and without any network requests.
      I_am_tiberius on HN
    • Such a shame it uses Vision API, i.e. it can not be replaced by some random self-hosted LLM.
      netika on HN
  • Hide window automatically after response

    From 1 comment
    • Love it! Will definitely use this when a quick screenshot will help specify what I am confused about. Is there a way to hide the window when I am not using it? i.e. I hit…
      hackncheese on HN
  • Windows version

    From 2 comments
    • This looks very cool. Does anyone know of something similar for Windows? (or does OP intend to extend support to Windows?)
      lordswork on HN
    • Very cool, would love to have a Windows version of this.
      mdrzn on HN

Questions

  • Whether calling it OSX versus macOS changes results

    From 1 comment
    • Did you find that calling it “OSX” in the prompt worked better than macOS? Or was that just an early choice that you didn’t spend much time on? I was skimming through the video…
      e28eta on HN
  • Whether the app can click and type on screen or run keyboard shortcuts

    From 2 comments
    • Hey, I was working on something to allow GPT-V to actually do stuff on the screen, click around and type, I tested on my Mac and it’s working pretty well, do you think it would…
      rchaves on HN
    • No, my thought process never really stretched outside of what I built. I had this particular idea, then sat down to build it. I had some idea of getting OpenAI to respond with…
      ralfelfving on HN

Alternatives named

  • Iris macOS GPT client

    From 1 comment
    • People reading this should check out Iris[1]. I’ve been using it for about a month, and it’s the best macOS GPT client I’ve found. [1]: https://iris.fun/
      kssreeram on HN
  • TypingMind

    From 1 comment
    • I have a hard time saying how much this particular application cost to run, because I use the Voice+Vision APIs for so many different projects on a near daily basis and haven't…
      ralfelfving on HN
  • Clipea

    From 1 comment
  • LocalAIVoiceChat and Linguflex

    From 1 comment
    • Was following these two projects by someuser on Github which makes similar things possible with Local models. Sending screenshot to openai is expensive , if done every few…
      Jayakumark on HN

Category, keywords, summaries and labels on this page were made by AI (Thread Reader v2) and not checked by hand. Each point links to the comments it came from.

Add a badge to your READMEShown on HackerShow
[![Shown on HackerShow](https://hackershow.org/badge.svg)](https://hackershow.org/show/38611700)