Open Tool Building

Tools here refer to any sort of software that augments a model’s capabilities by programmatically prompting and interacting with it.

Imagining tools, which I would consider as an affordance of openness, with potential to be merged into os - open source, openness.

• Imagining potential ways to use LLMs to build personalized foundation model tools (personal_tool_imagining, 42)

Implementing tools. There isn’t really a clear way to divide these into greater categories.

• Summarizing some content using an LLM. Though possible with online LLMs, people mention they use local models instead for this subsubtheme, albeit without rationale. (llm_content_summary, 17)
• Some kind of RAG implementation with an LLM. Highly related to ^ragcontextwindow6, though that is more focused on RAG being used specifically to reduce context window (llm_rag_implementation, 17)
• Using an LLM for playing a game or to be developed as part of the game. Related to ^llmttrpg3, in that language models are great when implemented within a roleplay-based game. (llm_in_game, 14)
• Using LLMs to write code (writing_code_llm, 16)
• Using LLMs to make stories or do creative writing (story_creative_writing, 12)
• Any form of fine-tuning an open model (fine_tuned_model, 14)
• Some combination of a model + software that augments some kind of personal assistant, like Jarvis in Iron Man (llm_personal_assistant, 10)
• Related to fun_experimentation, capturing use-cases that implement LLMs for some kind of entertainment. Could be slightly mistitled because many involve making friends out of LLMs. (llm_for_entertainment, 8)
• Google Photos clones, involving embedding images and retrieving them through NL. People prefer their local implementations because photos are personal data. Definitely related to privacy subsubthemes (vlm_image_recognition, 7)
• Using some kind of voice model as a part of the software stack (voice_model_integration, 7)
• Building web or local frontend for local or even cloud models (llm_frontend_building, 6)
• Using LLMs as some kind of bot on social media (social_media_bot, 5)
• Using a combination of OCR and local models to parse and process documents (reading_documents_ocr, 5)
• Using (small) LLMs to do translations. Some mention specifically on-device (machine_translation_llm, 4)
• Building knowledge graphs and retrieving from them using LLMs (building_knowledge_graphs, 4)
• Using LLMs to facilitate learning (llm_for_learning, 4)
• Getting feedback on some content from an LLM. Highly related to privacy/personal_documents, since most of the time the rationale for a local LLM is that the document is private material, like a CV (llm_content_feedback, 3)
• Applications of LLM function calling (llm_function_calling, 3)
• Building some sort of agents using frontier models (agentic_framework_building, 3)
• Peer-to-peer network of people’s computers to run models “locally” (peer_to_peer, 3)
• Using models to make embeddings of photos or other data such that it can be searched through natural language (natural_language_search, 2)
• Labeling using LLMs (llm_for_labeling, 2)
• Some combination of multiple models working in tandem. Mostly examples of image and language models (multiple_models_combined, 2)
• Testing multiple models on the same prompt to engineer prompts (llm_prompt_engineering, 2)
• Detecting phishing using LLM (spam_phishing_detection, 1)
• Using control vectors to change the behavior of models (control_vector_behavior, 1)
• Using LLMs with brain-computer interfaces (brain_computer_interfaces, 1)
• Building tool for inferencing open models (open_model_inference, 1)
• Using LLMs to generate text for search engine optimization (search_engine_optimization, 1)
• Some form of transcription or summary, or meeting companion implemented with an LLM and a voice model (meeting_transcription_summarization, 1)
• Doing some form of data science with an LLM (llm_data_wrangling, 1)