Sasha Stiles turned GPT-2 experiments into a self-writing poem at a Museum of Modern Art installation—and a new way to think about text-generating AI optimization ...
Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models ...
There's a lot you can automate.
We evaluate DeepCode on the PaperBench benchmark (released by OpenAI), a rigorous testbed requiring AI agents to independently reproduce 20 ICML 2024 papers from scratch. The benchmark comprises 8,316 ...
Your trading bot crashes at 3 AM because the forex feed went silent. Real-time currency data really shouldn't mean spe ...
Beta: This SDK is supported for production use cases, but we do expect future releases to have some interface changes; see Interface stability. We are keen to hear feedback from you on these SDKs.