LLM API Recommendation
Hello everyone,
I'm working with a nonprofit organization looking to use LLM APIs for translating and rewriting public domain books. We've explored options like TogetherAI, Anyscale, and Replicate, but their pricing (around $0.8-$1 per million tokens) is a bit steep for our budget.
Does anyone know of more affordable API options for accessing models like Llama 3.1 70B that would be suitable for our translation project? We're open to lesser-known providers or academic partnerships that might offer discounted rates for nonprofit work. Currently using loopin with their $50 free credits (Using code 5tSu) but we need much more volumes.
Any suggestions or experiences to share would be greatly appreciated. Thanks in advance for your help!
Comments
If you're talking about old public domain books, chances are there's also a public domain translation available. That would yield way higher quality translations, and lower the cost drastically.
Thanks for helping. We rewrite the story for 8-12 year old kids, breaking it into engaging chapters that are easy to follow. We'll also incorporate interactive elements, allowing young readers to converse with the book's characters.
Perhaps OpenAI GPT4o-mini using the batch API is an option? ($0.075 / 1M input tokens, $0.30 / 1M output tokens)
try deepinfra, I've been using them for 70B models but unsure if it's cheaper than you mentioned
Fuck this 24/7 internet spew of trivia and celebrity bullshit.
Use openrouter.ai then you can try various.
If you need cheap then deepseek is probably your best bet, though don't think it is particularly well suited to translation
Thank you for all the info!! We'll test above mentioned providers. GPT4o-mini is an ideal option. I'm not aware of their batch API before. For real-time inference, we'll use OpenRoute to evaluate various LLM models. So far, Llama 3.1 has proven capable. We tried DeepSeek's service, but despite being cost-effective, its performance fell short of expectations.
If time isn't a factor you could run GPT4All or llama.cpp locally
Don't have local GPUs.. API works better for us