With quite a few giant language fashions (LLMs), together with India-specific ones, being launched lately, Google-backed Slang Labs is choosing a hybrid mannequin to take the perfect from each LLM. It can even launch its personal model of some open supply LLMs within the first half of subsequent 12 months that can be area and India-optimised.

The firm affords voice assistants that may be embedded inside common apps like e-commerce or banks. The agency’s purchasers embrace Nykaa, ICICI Direct, Tata Digital, Fresho from Bigbasket, and others.

Currently, the corporate is utilizing OpenAI for its voice assistant. Kumar Rangarajan, co-founder of Slang Labs, mentioned they’ve began fine-tuning open supply LLMs like Meta’s LLaMA and French-based generative AI startup Mistral AI’s LLM to ultimately have a hybrid mannequin of LLMs for its voice assistant – CONVA.

“There are three layers for LLM – the primary is named the bottom LLM, the place it’s typically skilled with loads of web knowledge and completely different language knowledge for normal objective. This mannequin has an excellent understanding however not skilled to change into an excellent assistant. If you ask questions, it will be unable to reply them in a correct method.

While it has loads of information, it’s not that sensible to reply appropriately, as a result of it is rather poor in following directions,” Rangarajan mentioned.

Making base mannequin is an costly proposition as bulk of the price goes in them.

Making the bottom mannequin is an costly proposition as the majority of the price goes into them. The subsequent layer is the pre-training layer, the place the system learns about what’s the appropriate and what’s the incorrect reply. It learns to inform which reply to want when there are a number of solutions. There is a bunch of strategies to guarantee that the mannequin is ready to provide the proper reply.

The third layer is the fine-tuning the place the LLM has been skilled to reply correctly. It undergoes fine-tuning the place it’s made appropriate for particular use circumstances. “People like us or other companies like us can take this lower-level model and build and optimise it for particular purpose or use cases. We are taking base models from LLaMA and Mistral and pre-training and fine-tuning them,” defined Rangarajan.