Seeking to get other people’s experiences with finetuning.
Some people’s comments:
- Have gotten good results with fine-tuning for very specific tasks like translation, but not so good for story writing and roleplay.
- RAG can be a better alternative if the task is staying true to some set of given documents during generation.
- “In a lot of ways it’s kind of like making art with the concept of raw thought rather than the thoughts themselves.”
- First time I saw an extension to the offline from Reddit, “I’ll be presenting more on these ideas at the Microsoft Reactor in SF tomorrow, come by if you can!”
- The “ideas” being VLMs adapted to estimate spatial relationships and distances, and the general take that recent advances in fine-tuning have come from focusing on data quality over its quantity.
- “most of the way people finetune today is just a little bit more pretraining (same model architecture + loss functions), and so your additional data is pretty much drop in the ocean whose performance gains can be replicated by slightly smarter prompting. The real alpha we’ve seen is when folks are doing specialised, more constrained tasks - finetuning post modifying those two is EXTREMELY powerful - usually involves more classical ML tasks but not enough data to use BERT or something like that.”