PieFedOrbit by Mozilla
New Mozilla AI project. Put "trust" and "privacy" in the title and subtile but doesn't support locally hosted model.
Exists as an add-on today. Model is Mistral 7B hosted by Mozilla in GCP. Claims won't save data long term. Promises won't use personal information to train models and not share queries with Mistral or any other services.
Am I going to use it? No. Not without local model supported.
Note: the mobile version of the page is broken (lack of many content). Best to view the desktop version for complete details.
25 Comments
Comments from other communities
without sacrificing your privacy
Pls explain:
We also receive the following information from third parties when you use the Service:
Inputs and Outputs. When you use the Service to summarize or query web content, we automatically receive a payload back from the relevant [Third-Party Models] containing the contents of your query; information about the model queried (such as the name and version number); information about technical problems with processing the query, if any; the number of tokens required to process the query; and the model Outputs in response to the query. We do not store this data beyond temporarily caching it to process your query and return the Outputs to you.
Reminds me of the "summaries" from the 1975 Rollerball movie.
The characters in that corporate dystopia only had access to corporate-provided summaries of literature.
They couldn't access the original novels and works without high-level corporate authorization.
Gave it a try.
I don't like the floaty orb thing present on my window at all times. Even the "Minimal" option sticks around on the side. I like Kagi's extension more as it only opens after I click on it in the toolbar.
Gave it a try with a few YouTube videos and compared it against Kagi's universal summarizer.
Example: Scams In Software Engineering
The speaker in the video expresses frustration about being harassed to watch a video titled "Big Box" regarding scams in software engineering. They believe the video will mention them and criticize their opinions on clean code and software design. The speaker argues that software engineering is known for producing scams, but they're not referring to typical scams like Nigerian princes or crypto schemes. Instead, they're discussing the lies sold to developers. The speaker criticizes Uncle Bob's "Clean Code" book and philosophy, stating that it's a filler episode and that long functions with a clear behavior are more beneficial for easy maintenance. They also criticize Agile development and the idea of group decisions, believing it leads to the worst possible solutions. The speaker shares their thoughts on the value of a computer science degree and the importance of learning data structures and algorithms for solving most interview problems. They also discuss the concept of locality of behavior in software design and how it differs from clean code. The speaker expresses their frustration with the various scams in the software engineering industry, including the idea of mechanics needing to build a car from scratch to get a job, the overhyping of front-end development, and the use of Vim productivity scams. They also mention the high cost of keyboards and the importance of making something to begin with for good performance. The speaker concludes by thanking Brilliant.org for sponsoring the video and encouraging viewers to check out their website for learning math and science in a concise and fun way.Orbit's summary:
Title: Scams In Software Engineering Key themes include: Critique of Clean Code: The host expresses skepticism towards the principles of clean code, questioning the arbitrary limits on function lengths and advocating for "locality of behavior" in coding, which emphasizes understanding code through its context rather than strict rules. Agile Development: Agile is described as a flawed approach that often leads to mediocre group decisions, with the host likening it to communism in its failure to deliver effective outcomes. Education in Computer Science: The host argues that traditional computer science degrees are outdated, suggesting a need for more practical, trade-like schooling for programmers. He criticizes the core requirements of CS degrees, claiming they don’t adequately prepare students for real-world programming. Interview Processes and LeetCode: The video discusses the inefficacy of LeetCode-style interviews, suggesting they are a scam that doesn't accurately reflect a programmer's abilities. The host advocates for understanding data structures and algorithms instead. Frontend vs. Backend Development: The host humorously debates the perceived ease of frontend development compared to backend, asserting that both have their complexities and should not be dismissed. General Commentary on Software Engineering Culture: The video critiques the culture of over-engineering and the unrealistic expectations placed on developers, including the notion that everyone should learn to code, which the host argues is misleading. Overall, the video is a satirical take on the challenges and misconceptions within the software engineering field, blending humor with a critical perspective on industry practices and education. The host maintains a light-hearted tone while addressing serious issues, making the content both entertaining and thought-provoking. Time saved: 31 minutes.Kagi's summary:
The video titled "Scams In Software Engineering" features a humorous and critical commentary on various aspects of software engineering, focusing on perceived scams within the industry. The host reacts to points made by Big Box, a well-known figure in tech discussions, addressing topics such as clean code, Agile methodologies, and the education system for programmers.
While Orbit was faster in generating its summary, I much prefer the Kagi one as it appears to be more complete. Orbit had a few issues:
- Inaccuracy: BigBox is the name of the channel, not the name of the video.
- Lack of comprehensiveness: Orbit's "paragraph" style response format missed a lot of information that Kagi caught.
- Didn't catch the satirical/light-hearted tone of the video like Kagi did.
Needs work, IMO.
Another thing to note: Kagi requires an account but the summarizer otherwise is free. Orbit does not require an account.
I'm not testing it with emails as I would want the model to run locally to access personal data.
yea, you don't want 'ai' in your firefox.
so? *don't install it*.
it's an addon. you don't have it if you don't go and get it.
What I don't like is that money and effort were spent on developing it instead of other things.
Yeah but honestly this seems pretty neat. As an avid reader this is one of the last things I need for text, but having videos (which take forever to bring ridiculously little content across due to their nature, and most modern "creator" videos are just talking head shots) summarized as short text to read near-instantly fixes a big problem with the way content is dumbed down for modern audiences.
Is it weird to use a software to create that text summary instead of just having the original creator post their 2 minutes of actual content as text? Sure. Better than not having it available, though.
After trying it a few more times now that it's in beta, functionality is very good, I just wish it were integrated into the AI sidebar they're also working on instead of having its own floating button. minor issue of course.
For example their local translate. That always says: "And more languages are in development!" I think it'd be massively useful if I coud read Japanese. Or the Chinese Github comments. But I suspect Mozilla isn't really working on it but instead dabbling with other random projects like AI and this summarization, which I have exactly zero use for. I think we'd need massively better AI to do summarization. Even ChatGPT can't do it and this is just a waste.
I'm not mad that Mozilla decided to create a chatbot - especially since it's an optional add-on.
But it doesn't at all seem clear what the difference is between this and the countless other LLMs that are already available. I think they would have been better served working on other product features instead.
Neat project, but it's a bit odd that this extension that's focused on privacy doesn't allow you to use your own local LLM instead of connectign to their servers.
Edit: added a couple things to enable it on desktop (Linux)
Go to:
about:configSet
browser.ml.enable truebrowser.ml.chat.enabled truebrowser.ml.chat.hideLocalhost falseThen go to:
about:inferenceYou can set the endpoint to your localhost or any server.
Open sidebar and select AI chatbot.
Select localhost option and follow prompts
Note: I will never use this, I just wanted to know. Anyway, here are the docs
https://firefox-source-docs.mozilla.org/toolkit/components/ml/index.html
https://firefox-source-docs.mozilla.org/toolkit/components/ml/api.html
screenshots This is on nightly on mobile
Awesome, thank you
We discussed this briefly a few days ago. No one understands why Mozilla likes to waste their time and money on random sideprojects that nobody likes or asked for... Instead of something useful, or the things lots of people ask them to do.
And summarization is among the worst things you can do with LLMs. I'm not against AI, but they're really not good at this specific thing. I'm not sure if people will use it anyways, but I think this project is a waste of resources.
Literally every single non tech person I know uses these kind of tools. Everyone who searches google gets an AI summary of the results. Microsoft builds this into all their products.
Having an extension that users can CHOOSE to install to do content summaries is a good thing. Either Mozilla does it or OpenAI, Google, Apple, Microsoft do it. Personally id rather people be using a tool made by Mozilla.
its not like they arent focusing on their browser they just released a new address bar packed full of changes and feature to their beta branch.
Thank you. At least someone lives in the real world.
Whether Lemmy users like it or not, this *is* becoming an expected feature now, and Firefox shouldn't be exclusively chasing people on Lemmy who already use Firefox.
And I'd rather have it be implemented in a way that's pretty private, with the option of tying in a locally installed LLM (although it's a bit convoluted to do right now by the looks of it), and the entire thing be an optional extension, than it forced upon me.
My point is, we already have several local LLM tools and chatbots. This is just yet another one (which isn't ever there yet). I think you could as well use ollama for that.
While for example I still need to use Google Translate because Mozilla has a completely local translation tool for some time already. It's just they promised to add more languages, but they don't do it. Instead they use their time to get yet another addon to the prototype stage.
And you should try it. The (AI) translation is really good. It just needs a bit more polish and like 5 more languages... That'd help people massively. And it's also in demand, I heard Reddit and a few other platforms have added translation as well. If you want to help people and offer privacy, I'd argue you focus on that. And this would be something useful.
Summarization however, is not useful. I get people use it anyways. I just hope they have a look at the quality of the results. Because all I've seen are summaries that are between misleading and wrong. And that's by the market leading products like ChatGPT and Claude... I'm not here to dictate people's life. But they should be aware of it to make an informed decision. I think it's sad that Mozilla just has shiny advertising online for a product that has quite some caveats and is unlikely to ever work well. And I think misinformation is a big issue of today's world. It's marginally better to generate it while respecting people's privacy. Yes. But I'm not sure if that makes it a good thing.
And please continue working on the browser, the translation, Thunderbird and the dozens of other useful Mozilla projects. I think unless Mozilla has infinite money and developer resources, they should focus on products that work well for their users.
Yes, you *could* research local LLM tools, find Ollama, see if it's trustworthy, install it, configure it, research which model to use, download that, then run it. Or you could let Mozilla do the hard part for you.
The typical browser user does not go searching for GitHub projects.
Don't get me wrong, lots of tinkerers will do the above, and they still can. But this is a more user-friendly way for the average person.
I use an Ollama-based program on my PC called Alpaca (available on Flathub for any Linux users reading this), and it's pretty great and straightforward, but even that is more fiddly than simply installing a Mozilla extension.
And yeah I've tried Mozilla's offline translation, it's pretty great, I'm sure they'll expand the language list in time.
I wish I could agree, but I don't think I can. By that logic, Mozilla could as well stop developing their browser. It has dropped to a marketshare to like 2.5% by somewhat official statistics, maybe about 5% if we're generous. That's less than the ratio of Linux users. So I'd argue it's for tinkerers, too. Seems people aren't educating themselves, downloading Firefox, installing and configuring it either. I don't really know what to make of this argument.
But it's not my main point, anyways. It's been more than a year since the translation feature got added officially to the browser. They've promised to add more languages from the start. But we've only seen small changes since then, like how you can select text. Ultimately, that translation project is from 2022. I doubt they're actually working on it. I think it's a shame. We'd need some good AI tools.
They shouldn't stop developing their browser, and I'd never advocate for that. I'd have to go to Chromium, yuck.
I never said it wasn't for tinkerers, just that they need to attract people that aren't. And how does this move harm tinkerers? Not only can you tinker with this extension (such as by pointing it towards a local Ollama instance), but it's optional. You don't *have* to install it. What's anti-power user about developing this extension?
I'd also disagree massively with the idea of "well, they have a low market share, so they should forget about attracting more people and focus on tinkerers".
When you have a low userbase, you should seek to grow it, not simply double down on a small amount of people that already use your software anyway – especially not when, let's be honest, the tinkerer crowd on Lemmy and niche Reddit subs are the most fickle and hard to please bunch in the world. Mozilla could do everything they ask and that crowd would still complain.
Doubling down on a tiny amount of people is fine when you're Rolls Royce or Bugatti, and you can charge any amount of money to a small amount of people, but that strategy won't work for Mozilla. They need broad appeal, and they won't get that if they're lacking things that average people have come to expect.
Linux in the 90s and very early 2000s was impossible to use for any normie. Distros started focussing more on the average person rather than simply appeasing tinkerers who already use their software, and they've benefitted from that approach greatly – desktop Linux has never been in a better state! Why shouldn't Mozilla do the same?
I feel like Mozilla are in a difficult position. They're reliant on Google to exist, it seems. When they try to do something else to make an alternate revenue stream everyone says to stick to the thing they do that nobody in the world pays for.
I recently started donating to Mozilla. They have been delivering a good product for a very long time, the least I can do is pay for it.
I genuinely don't know what people expect from Mozilla.
People simultaneously want them to give up their search engine payments, but also get angry at them for trying to make revenue any other way.
Web engine development costs hundreds of millions per year. It's a phenomenally complex and expensive endeavour, with no obvious path to revenue unless you hoover up user data, which Mozilla doesn't want to do.
How is Duck AI then when it comes to privacy? I use it a lot.
What is duck ai
AI service by Duckduckgo which offers anonymous access to popular AI models like Chatgpt, Claude, Mixtral, Llama.
Could you run Mistral 7B on a consumer grade desktop ?
Yes. Depends on the actual hardware, parameters used, and model quantization, you can get 2~10 tok/s on CPU alone.
Consumer-ish. I can run it on my MacBook Pro, any 8GB VRAM nvidia card should be able to run it. Technically, any machine with 8GB system memory if you’re willing to run it really slowly
Well, at least Mistral is open source I guess :/
Does Mistral actually provide the training datasets, or are they using the fake definition of """open source AI""" that the OSI has massaged into being as megacorp friendly as possible?
AI should just be owned by the government and accessible to anyone.
It legit should be regulated like a utility, because it will become that important to being competitive.
I don't trust the government.
Should be a civil society or NGO.
Which government?