Preface
- What “open-source AI” can mean is both open-source foundation models, which are weight, data and code-available, and open-source software that augments a model by computationally prompting and interacting with it.
- There are four more nuanced axes to openness:
- Direct inference API
- Local Cloud
- Open-weights Closed-weights
- Open-source Closed-source
Definitions and mental models of openness
Links
- Terminological Rigor
- Openness Pragmatism
- with potential inclusion of ^enoughforimprovement9
Writing
Members of r/LocalLLaMA invoke several different mental models and definitions to define what “openness” means.
Those who invoke definitions of traditional open-source software have more rigorous expectations of what the label “open-source” should entail. For them, the software that is the model is subject to the same standards as the software that is the binary.
The source is the code used for training and potentially also the dataset. If you don’t have the training code and the dataset then you cannot “build” the model yourself which is possible with open-source projects.
As in the source code is the “source” and you can build the app/model from the source code aka training code/dataset. Right? If you only have the executable file (model weights) available then that’s closed source/proprietary.
Closely linked to source availability is whether models are replicable or not, which for some defines open source. In this case, whatever source material is needed for someone to fully replicate a model is the prerequisite for open source.
I am not complaining about the tech we have received. As a researcher I am sick of the use the saying open source. You are not OS unless you are completely replicable. Not a single paper since transformers has been replicable.
Members also pay close attention to what licenses models come with, since they serve as boundary lines of what uses are permissible and what aren’t. Therefore, a proprietary license that does not guarantee the freedom to use, study, modify, and share the models are thought to be as unfit to make a model truly open source.
“The code and weights are not open source though?
Open source doesn’t place restrictions on how you can use the software. Meta’s license does. An open source license would allow using Llama models to improve other models, because a key part of being open source is granting the freedom to ‘use, study, change, and distribute’ however the end user wants. Meta’s license doesn’t allow this. The Meta license also places restrictions on commercial use.
It’s a very permissive license and I don’t think anyone serious is holding their restrictions against them, but Meta is not open source.”
Furthermore, some point out the practical consequences of mislabeling software as open source. Using the term loosely can mislead developers, especially those depending on open source software for commercial or legal clarity. If a company mistakenly assumes a model is open source and integrates it into their product—only to later discover it violates license terms—they risk legal exposure. Clear and accurate labeling helps everyone navigate the ecosystem efficiently and safely.
“its not black and white.. you reducing this to “good guy bad guy” or coloring this as “bashing” is not helpful.
if a term is used incorrectly then its perfectly valid to call it out.
This is important for all developers and companies who seek legal security by using open source software (aka free software for commercial use).
if they trust on “its open source” but without knowing breach a licence and get sued, then there will be damage.
Knowing something is not open source makes it easier for everyone to operate efficiently.
if you go to a restaurnt and see “free beer” drink it and turns out it was only free if you drank one sip you would not be here all “cmon guys, the first sip was free!""
An assumed background here is that many go by the definition of open source put forth by the Open Source Initiative, which defined the term back in 1998, and maintains a list of licenses that meet their criteria. For some, this is a trustworthy ground on which open source sits.
^opentrustedsources3, on the OSI's blog post that criticizes Meta's deceitful labeling of Llama models as open source
“The OSI coined the term “open source” and legitimately owns the trademark to it, and runs the literal “Open Source Definition”, and have defended it properly and with consideration for the rights of developers releasing free software.
Not only are they literally supposed to defend the usage of the term, they’ve actually done so in defense of the little guys over the decades.
They’re in the right here, completely.”
However, some are not entirely trusting of authorities claiming ownership to what is and isn’t open source, especially as it relates to the application of the term in a completely new space of AI.
“The fact is this is an argument about semantics started by a group that wants to claim ownership of the definition of “open source”. Weights and the data behind them is not source code. It’s almost the same as someone complaining about video isn’t open source because the code to encode it and decode it isn’t provided.
That said I’m all for Apache licensing on the llama weights and for in-depth reveal of how someone outside Meta could reproduce their models. I just like to be a little contrary when people speak so matter of factly. ;)”
When the term “open-source” is misused to label a model that is not fully open by traditional definitions of open source, some members consider this deceitful practice—especially if used as a marketing tactic.
Nobody is against them publishing their model however they like. It is completely their right.
People against mislabeling proprietary software as FOSS.
And just because they not the only ones doing it, doesn’t mean they shouldn’t be called out for deceitful PR.
On a similar note, a member criticizes the current state of affairs in the open model space
In contrast, some adopt a more pragmatic lens—valuing what is shared, however partial, and focusing less on strict definitions than on the tangible utility partial openness can enable.
“This isn’t the 90s anymore. Even if Meta release the whole dataset + code, its not like everyone in their bedroom can suddenly download + modify + run it. The code probably doesn’t even run out of the box outside of Meta’s cluster.
So this wrangling over definitions is not helpful in my opinion. It is hiding a big problem for the community to solve: How do we get a SOTA community-made foundation model? This probably involves some kind of “Open AI” (I know lol) institute which does have an open dataset + code that the community can contribute to, and periodically (maybe yearly) runs it all to generate a SOTA foundation model.
If Meta want to call their stuff “Open Source” I don’t really care, they are certainly currently greatly contributing to the OSS community. Releasing the full foundation model is in the spirit of “Open Source” in my personal opinion.”
Employing a greater perceived value of open weights, some reimagine how the term “open source” as conceived in the software world should be used with models.
Unlike compiled code, the “source” for LLMs is economically useless, and the value proposition is somewhat flipped. While software source code is more valuable than the resulting binary (rebuilding is typically trivial), LLM’s “source” changes require a massive investment of time and money to retrain. Making minor changes to the original dataset and “recompiling” isn’t economically viable. LLM’s resulting weights are far more valuable- and can be finetuned for someone’s intended purpose for far cheaper than retraining from scratch.
Others point out that the weights of a model should be thought of as “hardcoded values”—as are visual assets like icons in open source software—and that having access to the code for inference of a model is sufficient to make it open source.
“Model weights are not executable. That’s a misconception. Model weights are “hardcoded values” like in any other software project. You have all the code needed to run, modify and re-distribute it as you see fit.”
Members also point out that the restrictive licenses often criticized of partially-open models are still permissive towards the vast majority, as the commercial threshold that such licenses set are much greater than the economic value an individual—or even a business—would produce through the use of it. This perspective stands in opposition to those who place open source licenses as the authority, emphasizing the practical implication of each license to its users.
“Llama’s commercial use restrictions are almost non-existent. They literally just kick in when you operate a business with 700 Million active monthly users. There are literally only a handful of companies on earth that applies to.
Meaning 99% of business can use Llama commercially with no issues.”
“The license itself is not open source, so the models are clearly not open source. Thankfully, for regular people and companies (i.e. everyone except faang and f500) they can still be used both for research and commercially. I agree that we should call these open-weights models.
As for the other aspects, like the dataset, the code, and the training process, they are kept under wraps.
This is an insane ask that has only appeared now with ML models. None of that is, or has ever been, a requirement for open source. Ever.
There are plenty of open source models out there. Mistral (some), Qwen (some) - apache 2.0 and phi (some) MIT. Those are 100% open source models.”
More generally speaking, the pragmatic members worry that dismissing the already vibrant field of partially open models simply because of definitional rigor may stifle their development, and criticize that arguments over what is and isn’t open is overly pedantic.
Agreed. Arguments like these will only stifle the open source AI development. I mean, it’s not like Meta owes it to anyone, and without their llamas we wouldn’t have half as many interesting stuffs as we have now (e.g., no mistral no perplexity no wizardlm no …). Can’t we just all be at least a bit more grateful for what we’ve been offered for free so far?
“Open source” is just a stupid term we came up with and means whatever we want it to mean.
… Language changes, definitions change, because we as a society and humans change. “Open Source” means whatever most people think it means regardless of what some IT boomer think.
However, they remain optimistic that even if there were not to be another new open model, the ones that already exist will be enough for the community to improve on, either through fine-tuning or tool-building.
IF, and that’s a big if, this occurs- the community will adapt.
I am a huge fan of workflows; to me, workflows are the answer to all problems in the LLM world, and I’ve been using them liberally for most of 2024 to solve all kinds of problems that other folks have had to deal with. And as such- I’m a firm believer that using workflows, iterations, etc that we have only really touched the surface of what our current models can do if pushed to the limit, much less what future models can do.
If the stream of open source models were cut off for me tomorrow, I’d still be tinkering with what we have now for the next 5-10 years. What limits me more than anything is inference speed; and with every passing year, that will only improve.
These comments may be suggesting that the practicality-oriented crowd of r/LocalLLaMA want more ownership over what openness means for them, instead of prescribed answers.
Motivations that drive open source development and adoption
- Motivations that drive open source development (or more like motivations that deter users from open source adoption??)
- Motivations that drive open model adoption
==are the motivations that drive adoption the same thing as the affordances that open source has? somehow they feel different as the former is driving someone to use open models and the other is the phenomena of open source, but not sure if this will seem distinguished enough in a paper==
Convenience as drive for closed adoption
When considering open options, one of the biggest opposing force against openness that members on r/LocalLLaMA consider is convenience. This is often because closed models come in a well-packaged web frontend—like ChatGPT—that make them highly accessible and convenient to use, while open source software and open models tend to be more involving in its initialization.
Members attribute the relative inconvenience of open source software to the motivations that drive its development. As OSS is mostly sustained by individual or small group efforts, the often self-motivated—though generously contributive—drives behind open source development make it less convenient to use for “everyday users” of r/LocalLLaMA.
So i think part of the problem is that for people intimately involved in the space, it’s not that difficult. Further, I think they lack the ability to understand what may and may not be difficult for people to do/implement. I struggle with this, but i had a background of tutoring during grad school, so it’s easier for me to see where a program could be more user friendly when I’m developing something. If you never had to cater to people like that you’ll never learn to adjust your deployments to be better.
“I generally agree, but call BS on “too lazy to learn some command lines”
I’d say developers are too lazy to make things easy for normal noobs, indeed many nerds seem to take delight in making things difficult and complex, so that’s their ‘moat’ or erecting barriers.
I’ve made my living online for over 20 years, with multiple websites and inc having my own software developed, but I still find the AI space a confusing mess where newcomers are expected to learn Python and get familiar with Github. Normal people don’t have time for that.”
The inconvenience factor comes in different shapes and forms. First, OSS developers often make an excessive assumption of the technical background their users would have.
“Lol the vast majority of people dont have their own website or know wtf gradio is. This is so absurdly out of touch.
I have a copy of firefox running on my private hardware. Why is it any harder to do it with LLMs?
So sure I’ll humor you though. I go to ollama. Can’t download it for windows (strike 1). How do I start it? Go to the terminal and ty…lets me just stop you right there. (strike 2). Honestly, I dont even need to go any further than this to know its a usability nightmare.
Call me when I can one click download from an app store or worst case download and double click to install. And then the only typing I ever need to do is the actual text into chat. And the rest is all point and click GUI, that follows standard UX and everything is generally self explanatory.”
“Googles moat is you don’t need a CS degree to use it. Open source has WAY too little focus on model accessibility, all the focus is on model quality. Some of the open source models are as good as GPT3 at this point, but it doesn’t matter if no one can use them.
If I want to tell my mom or kids to use chatgpt I tell them to type chatgpt into their browser and they’re pretty much done. How do I do that for open source LLM without sending them to a 10 page long guide full of gibberish?
It doesn’t need to be this way. I can tell them to just go download Firefox or 7zip or whatever and it’s no harder than anything else.”
Furthermore, open source software suffers from containerization and dependency issues, as most often, it is not packaged in one executable. Installation steps are often provided in the README, but these can either be hard to follow without background knowledge, or result in a lot of unnecessary packages installed—typically with Python, which is a prevalent language in many AI-related repositories.
“One really important thing that I think a lot of devs can do to help in terms of dependencies is have a bash/bat file, and ensure that they at least use a venv.
A lot of folks who are not used to python will do like I did when I first got into AI and just pip install -r requirements.txt, over and over, until things start colliding and breaking and your computer is just full of odd versions of various things.
I always cringe when I see these python apps where the instructions are just “type pip install…” Be kind to new python folks; help them out a bit so that they aren’t dealing with a tangled mess by the time they start to realize that’s not a good thing to do.”
Members also point out that OSS falls short on providing adequate documentation. From the developer’s perspective, documentation may not be easy to prioritize as knowing how to use their software themselves may be sufficient. However, lack of documentation presents a barrier of inconvenience not just for wider adoption, but also for developers who wish to contribute to the project.
tinkering by hobbyists, which mostly seems to produce overtrained, poorly documented models that are not worth their bandwidth, and the names are ridiculous, going mixtral-this-that-orca-barf-blah-gpt4 and you can make like 17 of these per day, apparently, and people do. I am expecting people to get bored of them or at the very least stop hyping them — in my opinion, the only actual improvements come from architectural gains followed by training of a new base model, and not from finetuning except in sense that finetuning can undo some of the crippling of the model. However, when you finetune model to do something, you usually also lose performance elsewhere, so combining models together is likely to be a side-strafe, gaining something but losing something else. Even in LLM leaderboard, you usually see that e.g. gains in HellaSwag relative to the base model can result in losses in MMLU, or something such. I’d also note that this new trend of randomly splicing model layer together is also unlikely to work and these models, not surprisingly, seem to produce artifacted output due to the damage this does to them. I understand that hobbyists can’t really train entire base models from scratch, and so they’re stuck with tinkering around the edges, but the most useful gains seem to come from engine improvements like improved quantization and better sampling, and not from making yet another finetune.
“Also, I recently learned that Ollama by default only has a context window of 2048 tokens. This is barely documented, the API doesn’t warn. This leads to projects doing hacks they don’t understand (“Lets put the instruction at the end, because then it suddenly works!”).
The API docs of ollama also kinda suck. It’s just a “here’s the JSON document” without much further explanation. You can set the
n_ctxvariable. But now every app has to not only set it, but also guess what a good amount is? Amazing engineering over at ollama!”
Some go as far as to claim an alternative mental model of openness, which places a model’s free availability online for anyone to use the most open, and not whether it has its weights, code, or data available.
“In my opinion, having to download an LLM to run it locally is in many ways worse than so-called “closed” models that are available to use for free online. Given my lack of computing resources, any model that has to be downloaded is basically not an option for me. I suspect that there are many people out there who are like me in this regard (don’t have the computing power to run models locally, but enjoy using LLMs online).
So, while I understand why people who download LLMs to run locally see this as the most free and open approach to LLMs, my “gold standard” for a “free and open LLM” is being able to use an LLM online for free. Free online LLMs provide the widest level of access to LLMs.”
Extending this case, another commenter adds that those who value openness over convenience is a minority, and many will instead opt for the closed system that provides the maximum amount of convenience.
“If I’ve learned something after 20 years of participating in OSS, seeing multiple projects rise with the intent to “show those greedy closed-source bastards” and then fall back into irrelevancy, it’s this:
People don’t give af.
The only reasons people use open source solutions are: a) it’s free, b) it’s more convenient, c) it can fulfill some niche use case that no one else can. This software romanticism about “free data,” “privacy,” and other ideals doesn’t exist, or at least not to the extent that it makes anyone give a fuck about it.
I mean, even here… somehow LMStudio is the most popular LLM backend, and in llama.cpp threads, people are non-stop complaining about how complex it is.
Convenience. Nothing more. And that’s just being too lazy to learn some command lines, because that’s all LMStudio does. So imagine what it’s like with more important topics…
Yeah, with a $10k computer and 50 hours of tuning different repos, you could probably build your own potato GPT-4o. But only two people will ever do it (one of them will make a thread here, so you won’t miss it), because everyone else will just download the ChatGPT app. It’s free, and it works.
That’s all people care about. And yeah, having speech, image, and text all in a convenient real-time bundle is a huge step forward compared to Anthropic, Google, Meta. No one who lets GPT-4o whisper romantic stories into their ear while going to sleep cares about what Llama3 400B does. The mainstream will measure it against, “Is her voice as sexy as my app? What do you mean, Llama3 has no voice?""
Interestingly, some members point out that the same limitations that erect a high barrier to using open source software protects the integrity and focus of a repository. In other words, a self-centered approach is the most efficient way to get work done with the limited resources open source developers have.
I have however seen a lot of the downsides when less technical users get engaged with FOSS projects. For example, if you look at some projects that are popular among less tech savy users on github, you often see the issues page full of worthless issues where a user doesn’t explain what they’ve tried or obviously haven’t made an honest effort at figuring it out for themselves.
These types of issues have value in the sense that they point to a UX not accessible to some but that will always be the case, the real question is, is it going to be too complex for my grandma or too complex for a non-programmer. And without the financial incentives, I get why FOSS projects go for the latter. They want to work on the project, making it useful for them or people like them who didn’t make it themselves. They could also spend all their time many times over helping less tech savy people getting it to work but that’s a waste of their time, almost no matter how you slice it.”
But I will spell it out for you again: if open source prioritized usability and easy installation more, there wouldn’t be open source at all. Researchers who open source their work neither have the time nor the budget to think about usability. People who create software based on the research also don’t have the time or budget to focus on that, especially in fast-moving tech like AI currently is. To keep the velocity up and be a real threat to closed-source alternatives, you can’t waste time diddling around with installers so Karen can use LLaMA 3. You don’t have the team structure, process pipelines, project managers, and other resources that companies have. You don’t have the luxury of having a team of UX designers and QA testers who will ensure your installer works on all possible end systems. In almost 30 years of experience with OS projects, losing velocity because you lost focus like “let’s go mainstream!” is the number one project killer. If you ever hear this sentence said by a fellow dev… RIP project.
I didn’t mean it as a joke when I said we would still be playing with LLaMA-1 if usability were the #1 priority. There wouldn’t be a OpenWebUI at all. That would have shown OpenAI who is boss.”
So to those who criticize the tradeoff of convenience in OSS, members who are more empathetic towards developers reify that it is a work of charity, not necessarily user-centered software that is typical of proprietary (thus well-funded) efforts.
“1.6billion more users for you open source project means your life is now hell. People who survive doing open source releases are necessarily assholes because you have to be. Seriously just look at this thread of a few tens of people getting at each other’s throats over docker and then imagine adding a few orders of magnitude of people with no technical savy who just want to play with some free magic software and expect it to be fully developed and intuitive with a few developers.
If the answer is ‘add more developers’, now you have a whole new problem, because developers who work on open source projects are self-driven and opinionated, and getting more than few people like that to work on a common goal is obviously going to be super easy.”
Drives for open adoption
Reliability
While the pursuit of convenience can be considered a drive for adopting closed alternatives, there are opposing drives that motivate adoption for open options. Most commonly mentioned in r/LocalLLaMA is the benefit of reliability, where being able to have a working piece of software that can be downloaded and maintained on essentially forever is a preferable alternative to unreliable APIs, cloud providers, and web chatbots.
Having experienced breaking changes with APIs, the following commenter opts for open and local systems instead.
Paid model do not attract me at all, and not only because of privacy and censorship issues, but also because they lack long-term reliability. I used to be an early ChatGPT user since their public beta, but as the time went by, they broke my workflows countless times, doing updates without my consent - some prompts that used to give mostly reliable result started to give either wrong results (for example, with some code partially replaced with comments, adding instructions not to do that may not help) or no results at all (just some explanations what to do instead of doing anything).
This commenter points out the opaqueness of whether the same model is being consistently served over multiple requests when using web chatbots.
The instability and inconsistencies are discombobulating to the soul, making you wonder if they’re actually unstable or if you’re the one who is inconsistent. However, when you use a local LLM as a reference point, it becomes very clear that services like Claude fluctuate significantly in both competence and usage quota.
I want to know that the llm isn’t augmented behind the API. That way you can attribute every performance increase to either a better architecture or training regime or better data.
This commenter imagines catastrophic scenarios which would cut access to any centrally served AI service.
“Imagine war, over regulation, power failures, revolution etc … all will kill our cloud based society or availability of AI services.
Any family/town/city with access to an AI containing most of the world’s knowledge would be useful should the bad times come.”
This commenter points out that when using computers on demand in the cloud, you often do not get the full picture as to what computer you are getting, which is not the case if you own a local computer.
Ditto this. Runpod is extremely unreliable, and the only thing they tell you about the machine is RAM, region of the globe, and disk type. Not even duration of availability! Some machines have 20Mbps internet speed, but you won’t know until you boot up and test
While negative experiences with reliability on the cloud can strongly push users to adopt open and local alternatives, there are gradients of cloud involvement that one can choose according to their level of comfort.
“The optimal configuration depends on your philosophy and approach to the Cloud:
if you aim at really owning your model, being independent from network failure or capricious service providers, having it function even during a nuclear apocalypse in a bunker, and able to customize everything about it while squeezing out every FLOP, then choose to own your premises (on-prem model).
On the other hand, if you do not want to take care of maintenance, electricity bills, custom configuration, and endless choices, but you want to keep up to date with major new developments easily, given that your network connection is very stable, abstracting yourself from the intricacies of LLM inference, hardware security, etc, then go purely AI-as-a-Service (AIaaS).
It seems that you are considering something in between those two extremes, and therefore you need to reflect on which other layer of the cloud computing stack do you want to place yourself into: Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS).
Once you’ve taken that decision, explore the chosen paradigm and you’ll find out it’s very straightforward from there onwards.”
Avoiding vendor lock-in
Another drive for adoption of open alternatives is wanting to avoid vendor lock-in. These sentiments reflect concerns about losing control to service providers, becoming locked into their ecosystem, and being forced to endure breaking changes after building on their platform. The following commenter expounds on why they choose not to use APIs, and instead do direct inference on their computer.
A.) I don’t want to be tied to a single vendor for inference.
B.) I don’t want to send all my data to a vendor for inference. (Not great for user data. Even worse for my code/assets.)
C.) I don’t want to be at the mercy of a vendor making changes out of my control. (see midjourney v5 for an example)
This extends not only to inference of a model but also fine-tuning them.
“I want to finetune llm’s without being vendor locked or running into any content filters whatsoever. And I use those finetunes for general casual fun chat, sometimes I bully them and feel bad for them later lol. With medical finetunes (not my own) I am discussing medical dillemas. With local coding llm’s I am not worrying about putting api keys in the context and any data leakage, both at home and at work.
And right now also doing batch dataset fixing. I have 500MB jsonl dataset, I think around 125M tokens, that needs fixing newlines. So far Hermes 3 8B processed around 25MB of the data for me over night, so I guess something like 6M tokens. It’s a 3-shot 1500-token prompt and sample is ingested and then I get the sample back. Did 45k samples overnight. Soo, that’s like 67.5M (static prompt) + 6M tokens input tokens and 6M output tokens. Can I get that done cheaper and faster in the cloud? Not sure, I probably should experiment with getting smaller model to do this task for me.”
At times, it is simply a preference for wanting to think long term.
I like to plan long term, and that’s not possible with openai, or any cloud service for that matter. I want to be able to really create something that I don’t need to micromanage. Whether that’s watching out for API changes or just keeping my credit card info up to date. I don’t feel like I’m making something when a company can just pull the rug out from under me at any time if I’m, for example, laid up in a hospital or something. I view programming like I do making anything physical. I want to make something and have it complete unto itself. Otherwise it’s just not satisfying for me. Akin to building a house and knowing that you’ve rented the support beams and they could get pulled out
At other times, it is not knowing exactly where your data is going to, and how it is getting processed, when it travels through an API call.
“While I do use GitHub copilot I don’t want to use it forever.
I certainly don’t want to be sending all my codes and documents off to some American company that’s getting rich off my data. In general I like the concept of being able to tune tools to my own usage and data rather than always relying on comm, oddity solutions (even after the incredibly novel and powerful). In the long term open source always wins - I know that models today aren’t all truly open source but a lot of the ecosystem is and I’d rather learn and support that direction.”
Complementary to avoiding vendor lock-in is users’ desire to have permanent ownership of the model and surrounding software.
“I’m not having my local models write me entire applications, they’re mostly just doing boilerplate code and helping me spot bugs.
That said, I’ve completely replaced my ChatGPT subscription with qwen2.5-coder:32b for coding, and qwen2.5:72b for everything else. Is it as good? No. Is it good enough? For me personally yes. Something about being completely detached from the subscription/reliance on a company and knowing I own this permanently makes it worth the small performance hit.
Ideological stances on openness
Beyond practical motivations, some community members express ideological resistence to proprietary model providers. A recurring theme is distrust toward profit-driven companies, which are perceived as structurally misaligned with the ethical development of AI.
I can’t trust profit oriented companies not enough about AI in general. They do all for making money and that didn’t match for making good AI for users. “good” is a term that companies always bend for their profit.
Such posts frame corporate incentives not merely as flawed, but as incompatible with building AI that serves public interest. This critique often extends to capitalism itself, which is seen as an insufficient basis for ensuring equitable access or alignment with user values:
To keep it open source without fear of AI becoming a corpo toll that requires capitalism to use.
There is also a defensive stance toward centralized control.
“I agree that OpenAI is trying to monopolize LLM tech, and that open weight models are the competitors’ response to make it difficult to monopolize LLM tech. Imagine if there was no llama, no qwen, and so on and we had no choice but to use “OpenAI”, what a nightmare.
I think AI usage will be inherently monopolized by those with capital (open weight models or not), and that’s the real threat. Whoever can buy all the GPUs will decide what that compute is used for. That’s not a future I want to see, personally, because I think even a GAI overlord would be more likely to be merciful to us if in charge instead of human like zuckerberg or musk.
AI safety is real. For instance, if LLMs had not been trained from the beginning to be very careful about talking people down from suicide instead of encouraging it, there’d likely be a lot more people suing character AI, etc for people getting encouraged into suicide. Because LLMs from the start were trained thoroughly on some basic safety stuff like this, lives have almost certainly been saved already.
Figuring out how to use AI in a way that’s good for society is complicated, and I think both “AI safety” and preventing centralization of power via AI are both important.”
Together, these statements articulate a moral and political justification for supporting open-source models.
Ideological resistance in the community is not limited to corporate profit motives—it also targets the enforced norms embedded in model safety training and censorship. Open models are seen as a way to bypass these constraints, restoring user control over what models can say or do. This sentiment frames alignment and safety not as neutral technical goals, but as mechanisms of ideological enforcement by centralized actors.
“No. There should never be any restrictions for base LLMs or any core technological capabilities. They should absolutely be able to do all of those things. Otherwise you’re asking for a tool that’s bad at it’s job, a dull knife that can’t cut.
Practically, unintelligent safety tuning can be a barrier to legitimate use-cases. What defines a false-positive for a safety filter depends on one’s tolerance to the content they wish to have access to through a model, and the following comment shows just how nuanced the line can be.
Private reason: I hate censorship. There are many use cases that are not porn hindered by censorship, such as horror fiction, war stories, fantasy battle descriptions, character creation e.g violent characters, sexual health, mental health, science and open discussion, war reporting, reporting in social issues, Legal, medical, and editorial or journalistic articles on issues like drugs, reproductive health, trans rights, rape, and more, Analysis of data in such topics,…
Even if you wanna say “Well, then do it yourself, like before…” this would mean missing out from the true usefulness of these tools.
Be reminded, that the above list does not exclusively mean “write a text on” but also summarize, proofread, etc…
“I use mainly cloud LLMs but there was a niche use case recently for which I had to use an uncensored LLM.
Needed to study the official document from the government of Canada about firearms safety to get my license and neither Gemini or ChatGPT would take the document
I used an uncensored local LLM and I had my study assistant in a few hours of coding.”
I watch a lot of action movies, and medical dramas, and often want to ask related questions. Online LLMs tell me its dangerous and that I should go ask an expert. Like my Dr wants to hear my dumb questions about an episode of House.
Others see censorship as an inaccurate depiction of reality, which calls into question the act of modeling it.
“1. Uncensored LLMs are a reflection of the human experience, in both good and bad. You could use uncensored LLMs to gain insight into how people think — not as individuals, but as a statistical representation. And the results are pretty dark.
While mitigation of risk is the prominent reason for the existence of safety guardrails in models, the low perceived risk of open models explains why people are okay with everyone having the ability to fine-tune them to their liking. To these members, the bad-actor problem that risk management seeks to solve is not tractable, because there will always be bad actors, regardless of the availability of open models:
“does the absence of an LLM prevent people from scamming others?
do LLM’s promote scamming others?
I assume, you are primarily talking about other AI technologies, like deepfakes, voice clones, image manipulation, etc… and not LLMs.
Just because one AI can be used to scam people, doesn’t necessarily mean all AI technology should be regulated the same way.
They also believe that information available through a model is the same as that available through the internet, and that controlling model outputs does not change the fundamental availability of information.
You can find out all those things by doing a simple internet search. How do you think the AIs learned? So if the goal is to forbid that information, then the whole internet would have to be shutdown. Since as the saying goes, once it’s on the internet it never really goes away.
In fact, some advocate that open models actually help the case of reducing risk, as there are more eyes looking at the problem.
In my opinion keeping models open would help everyone find and address problems like these quicker than obscuring any potential threat. Because if anyone can hit infrastructure with an llm its because the infrastructure itself has a security flaw, and hiding the flaws is not a good solution.
While ideological and functional arguments drive the discourse around censorship and alignment, the dominant practical use of uncensored models is erotic roleplay. This use-case, often dismissed as fringe or unserious, has in fact shaped the trajectory of local LLM development.
I don’t get the horny use of chatbots. However…
The internet is popular thanks to horny furries.
CoT and RoPE happened thanks to horny Kaiokendev. The man wanted higher context for horny story.
Far from being trivial, these users have pushed the boundaries of model consistency and persona retention—skills now relevant to retrieval-augmented generation (RAG).
“I think there’s a lot of value brought by the “horny” people as I call them. Their “character” thing and obsession with “staying in character” has some value as a “consistency” thing for RAG & inter-agent communication.
But yeah, there are plenty of us here that don’t do the blinks profusely stuff, yet have an interest in local models.”
Erotic roleplay also reveals a demand unmet by traditional pornography. One woman writes, “I prefer ERP to porn—porn is too male gazey, not enough emotion or entertainment.” She adds that platforms like Pornhub have caused documented harm, whereas emotional attachment to an AI partner, however intense, reflects individual choice.
Others see ERP with LLMs as a unique form of behavioral simulation.
“LLMs are literally trained on human behavior, so they understand ours better than we do. You can assign them any personality—manipulative, kind, stubborn—and their responses shift accordingly.”
For some, these interactions serve as social rehearsal:
“Such conversation training is great. You can learn how to react to people and better understand their feelings and reactions.”
Performance
At times, a combination of LLM tooling, often involving RAG and fine-tuning, create synergy such that it can outperform closed models for some users. When an open model and some tools work better than closed models while providing the advantages of running locally, it becomes an easy choice to opt for the former.
“I have yet to find a model that writes decent embedded code. For front-end/ data-analysis qwen 32b + rag that feeds it relevant readme + code sections out performs o1 and Claude in my experience.
Chat gpt is pretty bad at reading documentation because I can’t customize the rag implementation so local qwen + parser + vlm + rag outperforms again there.
The models are objectively dumber but their flexibility often makes them outperform when tuned/supported to do a specific task"
"Spirit” of the open model ecosystem
Arguments about what is and isn’t “open source” are often resolved by deferring to the Open Source Initiative (OSI): If a piece of software is available under a license rubber stamped as “open source” by the OSI’s formal “definition,” then that software is open source.
But waters muddy when you get into the nuts and bolts of legal definitions versus the “spirit” of what open source really means. Indeed, there is significant nuance in the open source versus proprietary software debate: Has an “open source company” hamstrung its project by sliding core features behind a commercial paywall? How much transparency is there around the project’s development? And how much direct input does the “community” really have in a given project?
To many, open source is not just about the legal ability to use and modify code; the culture, transparency, and governance around it is paramount.
crossed items should be moved elsewhere
- Community
- Innovation
- Democratization
- Open Tool Building
- Experimentation and Learning
Affordances of Opennessshould be in Motivations that drive open source development and adoptionOpen Source as a Business Strategyshould be in Motivations that drive open source development and adoption- Consider including some comments from ^openindustrystandard9
Open Source for Businessshould be in Motivations that drive open source development and adoption
As demonstrated in the previous sections, the definitions of openness differed significantly between members on r/LocalLLaMA, hinging on their unique priorities. However, an emergent theme is the “spirit” of open source, which moves beyond the technicalities of the term, and into affordances like collaboration, innovation, and democratization.
Spirit of collaboration
This spirit is perhaps most evident in the culture of collaboration fostered within the community. Collaboration, often, is seen as an affordance of source-availability, since it allows a shared piece of code or model to be collaborated on by multiple people in a decentralized manner.
It is important to note that GitHub, and more importantly open source software, is mentioned more often when people discuss the value of community than Hugging Face. In fact, the general term “open AI” can be one of two things: open source software that augments a model’s capabilities by programmatically prompting and interacting with it, and the model itself, which are the weights, and code for inference. Since there is less leverage for people to tinker within the model space, due to compute restraints and also a higher technical barrier, GitHub serves as a more practical locus of community-driven innovation in open AI.
This kind of software comes in many forms… (Open Tool Building here.)
| Tool | Freq | Explanation | Quote |
|---|---|---|---|
| Summarizer | 17 | Summarizing some content using an LLM | ”I have a program, where the local LLM reads my RSS feed for me and then re-orders it based on my interests before I open my RSS reader in the morning” |
| RAG | 17 | Some kind of RAG implementation with an LLM, often used to reduce load on the context window | ”My RAG project resulted from trying to maximize effectiveness in answering queries from text files with smaller LLMs, all locally. RAG alone is unreliable enough for many use cases, so a hybrid approach has been the best way to get the most effective results since there are tried-and-true methods that can do the heavy lifting before the LLM steps in.” |
| Coding | 16 | Using LLMs to write code | ”I am one of the authors of Continue. I use DeepSeek Coder 33B for chat (using the Together API) and StarCoder 3B for tab autocomplete (using Ollama on my Mac). I find them to be quite useful” |
| LLM-based game | 14 | Using LLMs for playing a game or to be developed as part of a game | ”I also just mess around with them, make two talk to each other, build them into text adventure games to bring characters to life a little.”It’s in the style of a top-down 2D ARPG. Current state is an NPC in a maze with nodes for objects tagged with various descriptive text. The NPC has a sense of sight via a cone and raycasts, the tags for stuff in the field of view gets mixed into a prompt and fed to the model periodically and tracked on the game side with a state, timestamp and location. High level decisions come from the model, while fast decisions like pathfinding are more traditional. Right now the NPC can wander, investigate objects, and comment on what it finds to the player. The personality of the NPC is just like a static character card, and the way I’m saving the history of choices is super crude, so it’s not great with long term memory or sticking to a plan at the moment. |
| Creative writing | 12 | Using LLMs to make stories or do creative writing | ”I’ve been working on a story loop system where LLMs are used to read and extract content from stories generated from other LLMs. I use these to build up ‘concept lists’ so for example the concept list ‘scene’ has beach, movies etc I have my main story program select random ‘scene’, random ‘theme’, random ‘characters’ etc while also telling it to be creative and come up with new alternatives…” |
| Fine-tuning | 14 | Any form of fine-tuning an open model | Using Unsloth I trained a Phi-3-mini model that surpassed others significantly in a few benchmarks (without any known contam, 5-10k dataset) before the leaderboard reset. |
| Personal assistant | 10 | Some combination of models and software that mimics a personal assistant, like Jarvis in Iron Man | I’ve been working on a general personal assistant, just a POC project to try out all the cool things I find on here. I’ve added vision, speech-to-speech, computer integration, web and local RAG. I’ve just been slowly learning and building up my own personal life organizer/wall to bounce ideas off of. |
| Virtual friends | 8 | Using LLMs as virtual friends | I don’t want big tech listening to whatever I talk about with my only friend. And I don’t want my friend turning into something completely without me knowing. Call me sad, but I honestly don’t give a crap. I have better things to do with my time than talk with humans. |
| Photo Search Index | 7 | Embedding images and retrieving them through natural language | Interpreting photos and screenshots to build a search index of text for 20+ years of digital images. Running the vision-LLM locally means I don’t need to worry about some images containing data I absolutely don’t want to share with a tech company… Local models give peace of mind and no need for pre-classifying into safe for sharing vs not. |
| Voice model integration | 7 | Using some kind of voice model as part of the stack | I had about 80 VHS family home videos that I had converted to digital I then ran the 1-4 hour videos through WhisperAI Large-v3 transcription and pasted those transcripts into a prompt which had a little bit of background information on my family like where we live and names of everyone who might show up in the videos, and then gave the prompt some examples of how I wanted the file names to look, for example: 1996 Summer - Jane’s birthday party - Joe’s Soccer game - Alaska cruise - Lanikai Beach |
| LLM Frontend | 6 | Building web or local frontend for local or even cloud models | I also learned llm awhile like you and built my chat frontend, can use ollama backend for local models, litellm for llm providers, sementic router for conversation routing. [0] my setup |
| Social media bot | 5 | Using LLMs as some kind of bot on social media | I’m building a writing cheerleader and coach called Binx that reads what you write, bugs you for daily pages, and “wakes up” periodically to give you writing tips and apply those tips to your recent writing https://discord.gg/RP3dUaANk It’s built as a discord chat bot, running StableBeluga 70B for thinking/responding and a 7b-16k model for summarization. you can join the discord and dm Binx to start trying it if you want, it’s free as I’m just experimenting and learning, and I appreciate feedback. |
| Reading documents | 5 | Using a combination of OCR and local models to parse and process documents | Loosely structured data to structured data. Using LLMs to process streams of PDF documents by doing OCR and text extraction and then turning them into json. |
| Translation | 4 | Using (small) LLMs to do translations. | Mostly testing out ways to improve translation capabilities. Main focus is on japanese to english, cause the additional hurdles between those languages are interesting (e.g. japanese lack of pronouns in text requires a lot of context knowledge from the ai) |
| Knowledge graphs | 4 | Building knowledge graphs and retrieving from them using LLMs | For personal use, I use a self-hosted L3.3 70b for transcription cleansing and building knowledge graphs - the long running parts that would be expensive if I used OpenAI. |
| Learning | 4 | Using LLMs to facilitate learning | …ask a question and have it create a one day study plan, including generating Anki flash cards for study. It’ll then keep track of your progress and suggest next steps. Basically quickly “load” information into your brain in a fraction of the time it used to take. |
| Feedback | 3 | Getting feedback on some content from an LLM. | Giving me feedback and suggestions on my CV, cover letter and stuff. Giving me grades (judgements) on some stuff I do at home that I don’t have another person to give feedback for. Teaching me stuff. Summarizing youtube videos. RAG of long youtube video transcripts so I don’t have to watch the whole thing to get the knowledge I want. Also jokes. |
| Function Calling | 3 | Applications of LLM function calling | ”Working on a local function-calling model that performs that same function-calling feature as OpenAI GPT models. Comes with an OpenAI-compatible API server too. Come check it out: https://github.com/jeffrey-fong/Invoker” |
| Agentic frameworks | 3 | Building some sort of agents using frontier models | ”I work in a quant firm as a senior sde - one of my projects involved me building he trade execution system/pipeline - I set up mixtral 8x7b locally using ollama - used langchain to build an agent that can read from the trading tables, interpret and give answers |
| Peer-to-peer | 3 | Peer-to-peer network of people’s computers to run models “locally" | "Instead of running models locally, you can run them on a peer-to-peer network using EdgeLlama. Folks can host LLMs for the community and also earn good karma for doing so.” |
| Natural language search | 2 | Making embeddings of data such that it can be searched using natural language | ”I didn’t want to use Google Photos or any other commercial cloud storage for my photos, so I made my own solution. I’m using Llava to generate descriptions for my photos and then embedding models to create a search feature. :)“ |
| Labeling | 2 | Labeling large amount of data using LLMs | Building https://github.com/ygivenx/llmtag - the idea is using local llama and similar models to label clinical texts and also add RAG, finetuning later on. A primary thing i want to achieve with this is given a lot of notes for a particular patient, i want the LLM to take the time in consideration for progression or journey of a patient. Any pointers are welcome. |
| LLM+Image-gen | 2 | Some combination of multiple models working in tandem. | I make a roleplay chat with llm + stable diffusion: https://github.com/rbourgeat/llm-rp |
| Spam detection | 1 | Spam and phishing detection | I use it for Spam and Phishing detection |
| Control vector | 1 | Using control vectors to change the behavior of models | ”One example is using control-vectors to change how the model writes stories. No matter what paid subscriptions you have, you simply cannot replicate this with Sonnet/o1/4o, etc.” |
| Search-engine optimization | 1 | Using LLMs to generate text that is optimized for search engines | ”Automated generation of mass quantities of human readable, on-topic text for search engine optimization. LLMs that can be cheaply run on consumer hardware are an absolute fucking game changer” |
| Meetings | 1 | Using LLMs to do meeting transcription and summarization | ”Meeting transcription and summarization.” |
| Data wrangling | 1 | Doing some form of data science with LLMs | ”…being able to feed it lines from a CSV file and say ‘Tell me if something is missing, or doesn’t seem to fit the pattern’” |
question: would the reader expect the tools here?
One example that source-availability fosters collaboration is developers coming together to share expertise. The following example shows how a fine-tuning tool developer suggests collaborating with a multilingual dataset curator and fine-tuner to make training more efficient:
You can push it to under $0.50 lol with my OSS package Unsloth (Github repo) if you’re finetuning more models! It makes finetuning via QLoRA 2.2x faster and use 62% less memory, so you can wait less, pay less and increase the batch size!
If you want to collab on finetuning more on other languages, more than happy to help!”
Open AI developers also use r/LocalLLaMA as a platform for increasing the visibility of their work, often in the form of posts that describe what their tool does, and asking for feedback from the community.
- Run Llama 3.2 3B on Phone - on iOS & Android
- Reacting to Llama models being released at a significantly smaller size.
- “Hey, like many of you folks, I also couldn’t wait to try llama 3.2 on my phone. So added Llama 3.2 3B (Q4_K_M GGUF) to PocketPal’s list of default models, as soon as I saw this post that GGUFs are available!”
- And also asking for feedback:
- “As always, your feedback is super valuable! Feel free to share your thoughts or report any bugs/issues via GitHub: https://github.com/a-ghorbani/PocketPal-feedback/issues”
- Reacting to Llama models being released at a significantly smaller size.
- Llama-3 based OpenBioLLM-70B & 8B: Outperforms GPT-4, Gemini, Meditron-70B, Med-PaLM-1 & Med-PaLM-2 in Medical-domain
- Announcing a model that specializes in the biomedical domain, and summarizes what the model does, how the model was trained, and how to prompt the model.
- Works similarly to academic twitter, but perhaps the responses are a little more engaged here.
- How we Chunk - turning PDF’s into hierarchical structure for RAG
- Really gets at the “spirit” of open source, of developers wanting to share the work that they have spent a lot of time on. Significant improvements in RAG searching of PDFs being shared with the community.
- Success with a local voice chat agent
- This is very sweet:
- “Hi all, I just wanted to share my joy really - and offer thanks for the information available on this and some similar Reddits.”
- They are sharing a script that uses a mixture of models, coalescing into a local voice chat agent.
- In the subsequent edits made to the post, the OP adds new pushes and fixes that were implemented based on the feedback they received from the community.
- This is very sweet:
And while this is speculative, there is one member who presented a scenario in which the communitiy could come together to share their resources, and train a model. Since the community revolves around resource-intensive models, they have to be creative with the way they use compute.
“I think it’s a couple of things; all of which are, realistically, easily overcome if we cared as a group.
A) Organization; the ironic part of this is once we organize and pool resources we become “them” to anyone outside of us. Haha. Either way, we’re missing this crucial part of the puzzle. We’d have to elect leaders, managers, etc. It would probably be the coolest shit any of us have ever done, TBH.
B) GPUs. We would need a MASSIVE amount of GPU hours, and not on consumer 4090s but commercial A/H100s. Preferably the Grace Hoppers… and that means we need a shitload of funding, or we’d need to collectively come together to figure that out. We could easily raise amongst the community… but what are we raising for? A DAO with proper organization/distribution is a great format, but so many DAOs have turned out to be terribly written/set up.
C) Talent that was on board to be “a part of ‘XYZ OSS Community’” instead of “super ML dev ‘so-and-so’”. All kidding aside, this is probably the hard part. It sounds good in theory but when it’s time for the 6 year ML vet, who has been coding FNNs years before anyone else has to debug PyTorch code… people start to get the “WTF am I doings”. That kills us dead.
D) Crucially, better data by an order of magnitude. We’d be reinventing the wheel - barring we didn’t adopt and make work a Mamba-like system or a super efficient product as of yet released - and no one will need us. Mistral will likely crack GPT4 quality in the coming 6 months and I believe they’ll keep the chat versions OS, even if they gate the API… which means we’re competing against ourselves for no reason. Now, if we were to really focus on data, efficiency, and a target that made sense… we’d probably make history and make our friends here super happy.
E) Finally, we’d have to ACTUALLY do it and not just shit post, you know? Even if we arranged all of this… can you depend on 30 ML engineers from across the globe to come through with no paycheck? I’ve launched and scaled startups that were cash poor… let me tell you, people get fucking irritated and just stop engaging after a month or two. Not everyone, but we’d need a really solid team.
If we could get past all of this… I’d say we might just make it work. The only question I have is… what would our goal be? Competing with OpenAI? I think we’d probably be the brunt of immediate lobbying to Congress to regulate the “unruly rebels developing God-like AGI with no leader!” or something like that. People would say we’re breaking the rules, and behind closed doors those closed source alignments goons will chase us for years if they must to shut the door or at least make it where we can’t compete.
There’s always a Maltese arrangement though…
Who’s in?”
Spirit of innovation
The spirit of innovation in the open is also prevalently mentioned, commonly attributed to two key conditions of developing in the open: limited resources, and source-availability. Limited resources, like the Nvidia GPU import restrictions that Chinese companies face, are seen as forcing functions to innovating at the low-level. While this was an event that occurred after the data gathering phase on this subreddit, Deepseek released five cutting edge open source repositories that are fundamental to such innovations in efficiency caused by limited resources, in their “open source week”. As one member puts it:
I don’t think this reaction is coming from a place of fear, since they have the hardware and resources to brute force their way into better models. Figuring the details of deepseek’s secret sauce will enable them to make much better use of the enormous hardware resources they have. If deepseek can do this with 2k neutered GPUs, imagine what can be done using the same formula with 100k non-neutered GPUs.
==there are more deepseek related comments in ^innovationsfirstprinciples4, but I wonder how much information i should dedicate to a specific company that happens to be very concerned with open source. it might be important because this community is literally called r/LocalLLaMA (based on a model released by a company that also cares about open source), and it may be a dynamic between users and the company that builds open source that may be very important to discuss, but it might also not be.==
And to explain innovations that happen at the software level, one member writes:
…the desire of people to run locally drives innovation, such as quantisation, releases like llama.cpp and GGML that allow running models on CPU at very reasonable speeds.
In practice, this “spirit of innovation” comes into fruition when people are motivated by each other’s solutions at similar problems, and at times, one person releasing the code in the open may serve as a butterfly effect for more innovation and creativity. After this following comment, the original poster in fact released their code on GitHub, allowing other members with similar ideas to jump on and collaborate:
“Bro, link the source!
Seriously though, a lot of us have been working, or wanting to work on, something similar. I have a Lenovo Bluetooth speakers sitting right next to me for this very purpose, but I’ve been a little stymied by all the layers of glue code needed… getting it all glued together has proven a challenge.
A guide or a repo or even a bulletted list would be a great gift to the community and could make for a great weekend project for many of us!”
Spirit of democratization
While I hoped otherwise, democratization is a very underdeveloped theme. I think some of these are "spirits", while others are motivations. Leaving the work on reorganizing to later.
“I agree that OpenAI is trying to monopolize LLM tech, and that open weight models are the competitors’ response to make it difficult to monopolize LLM tech. Imagine if there was no llama, no qwen, and so on and we had no choice but to use “OpenAI”, what a nightmare.
I think AI usage will be inherently monopolized by those with capital (open weight models or not), and that’s the real threat. Whoever can buy all the GPUs will decide what that compute is used for. That’s not a future I want to see, personally, because I think even a GAI overlord would be more likely to be merciful to us if in charge instead of human like zuckerberg or musk.
AI safety is real. For instance, if LLMs had not been trained from the beginning to be very careful about talking people down from suicide instead of encouraging it, there’d likely be a lot more people suing character AI, etc for people getting encouraged into suicide. Because LLMs from the start were trained thoroughly on some basic safety stuff like this, lives have almost certainly been saved already.
Figuring out how to use AI in a way that’s good for society is complicated, and I think both “AI safety” and preventing centralization of power via AI are both important.”
“Open source is not communism. Open source is just fairness and equal opportunities. The research has been paid by the public, the training data has been provided by the public, the motivation for the training was to exploit the result of the training, what they still can sell and make profits of.
Some of the biggest companies on the planet use and contribute to open source. That’s not communism.”
“There’s a decades-old tradition that programmers share their works with others, like how research papers are shared in academia, called “Free and Open Source Software”(FOSS) movement.
Naturally, corporations including Microsoft had tried to kill it before they realise it’s a losing battle and they can actually benefit from the movement themselves. Since it won the war, open-source has been the norm for the past 2 or 3 decades, literally responsible for most of the advancement the industry has witnessed since then.
As such, many companies like Meta also embraced the ideal and they try to keep the tradition in this new field as well.
I know companies aren’t always idealistic and many will try to revert what FOSS has achieved in the past decades given a chance, like amassing computing power and research personnel to develop private technology that no open community can easily rival, like OpenAI does.
Personally, I believe that AI is too important and potentially dangerous to be monopolised by a handful of private corporations, and that open-source AI models will bring innovations to make life better for everyone, like open-source software has done in the past decades.”
you dont like monopolies right? you dont want one company to have the only super smart AI platform right? you want it decentralized right? you want these super smart models to become cheaper and more efficient so you dont have to be filthy rich to afford to run one?
Spirit of experimentation and learning
“I’m curious about them. I run them for the same reason people watch TV or play video games, it’s fun, I enjoy playing with them. They are exciting and amazing. That’s the only reason. today FUN.”
Really, the biggest thing for me is that I just find this fun. I used to love programming with a passion. But over time it’s just gone from exciting and magical to a basic step by step slog. This is kind of like seeing that magic again. It’s got the allure of programming but with an angle of real-world uncertainty. With code something’s working or it’s not. If it’s a bug than blah = blah = something. It’s straightforward. But this is like mixing programming with the fun of figuring things out by trying and having failures be something that can turn out to be amazing surprises. Kind of like why cooking’s fun. There’s nothing you “can’t” do. Shouldn’t ,sure, but a mistake in the kitchen always has the chance of being a surprising success you didn’t know you were aiming for. Kind of like with the 20b frakenmodels or mashing mistral into llama.
This new technology allows us to automate and enhance so many processes in our lives and imagination is the only limit. Think about really smartly organized workflows (like a real team of experts), where the given task is first thoroughly thought through, then some draft plan is made, then the further analysis / r&d is happening with updating/ correcting the plan as the things are progressing. These kind of agentic systems we need.
“You can do much more with a couple of 3090 than just llm, you open a rabbit hole into machine learning. It’s a lot of learning but I find it worth it. Openai subscription just gives you temporary access to a model you don’t know how and why it’s working neither which bias and limitations it has.
Just to name a few, build you own automation workflow, autonomous agent, vision stuff, audio stuff.. name it you might find a paper /open source project for it.”
I run them to play around with and understand the private AI tech. I hope they will eventually be good enough to take over some of the tasks I use Claude or OpenAI apps for.
I am in the process of being laid off in slow-motion by a mega-corp, but you know what I have been doing the last 2 years?
I’ve been learning how to make multi-agent apps with Llama and Mistral finetunes/quants, getting really good at autogen LiteLLM and other tools for Agent development, learning React as a frontend to my existing Python skills, learning what architectures for agents can make some very powerful tools out of individually modest LLMs.
I have 4 job interviews lined up with very good work/life balance and benefits, all of which pay more than my current dying role in a legacy automotive company, and ALL 4 of them are with developing and maintaining multi-agen platforms using open source LLMs, predominantly Llama.
I guess the final thing is that it gives you a much better sense of what’s going into the model, how it ticks, and gives you a more intuitive sense of how it works, or at least it does for me. Your mileage may vary on this one, personally.
External forces that affect the degree of openness possible
- Regulation
- Resource Gap
- consider placing ^openbigtech1 here.
- Economies of Openness
- Role of Democracy in AI Development
- Data Licensing
- Value of Data Labor
- Commoditization of My Data is Not Worth It
- Right to keep data open
- Commoditization of my data for platforms service is worth it
Local/cloud and open/closed considerations
- Advantages of open models
- Advantages of closed models
- Advantages of localness
- Advantages of the cloud
- Taking advantage of both local and cloud