Show HN: Prompts as (WASM) Programs https://ift.tt/iPz261G

Show HN: Prompts as (WASM) Programs AICI is a proposed common interface between LLM inference engines (llama.cpp, vLLM, HF Transformers, etc.) and "controllers" - programs that can constrain the LLM output according to regexp, grammar, or custom logic, as well as control the generation process (forking, backtracking, etc.). AICI is based on Wasm, and is designed to be fast (runs on CPU while GPU is busy), secure (can run in multi-tenant cloud deployments), and flexible (allow libraries like Guidance, LMQL, Outlines, etc. to work on top of it). We (Microsoft Research) have released it recently, and would love feedback on the design of the interface, as well as our Rust AICI runtime. I'm the lead developer on this project and happy to answer any questions! https://ift.tt/XLqAdPY March 11, 2024 at 10:30PM

Comments

Post a Comment

Popular posts from this blog

Show HN: Agent File (.af) – An open file format for agents https://ift.tt/fzI5HcG

Show HN: Agent File (.af) – An open file format for agents Hi HN - We’re building Agent File (.af), which makes it possible to re-create the exact same agent (including memories, tools, message history, configs, etc.) across different machines. A big difference between LLMs and agents is that agents have associated state: system prompts, editable memory (personality and user information), tool configurations (code and schemas), and LLM/embedding model settings. While you can run the same LLM as someone else by downloading the weights, there’s no “representation” of agents that allows you to re-create an instance of an agent across services. Agent File (.af) is an open standard file format for serializing stateful agents. Originally designed for the Letta framework, .af is a human-readable representation of all the associated state of an agent to reproduce the exact behavior and memories. To demonstrate .af, we also made a few example agents with download links to .af: - MemGPT: An agen...
Read more

Show HN: Sort lines semantically using llm-sort https://ift.tt/7vEeHKP

Show HN: Sort lines semantically using llm-sort This is a small plugin I made for Simon Willison's llm utility. You can do things like: cat names.txt | llm sort -q "Which one of these names is best for a pet seagull?" cat books.txt | llm sort -q "Which book is more related to basic vs. advanced CS topics?" I see a lot of potential marrying LLMs with classic UNIX interfaces. https://ift.tt/9GDceOu February 11, 2025 at 08:55AM
Read more

Show HN: AgentKit – JavaScript Alternative to OpenAI Agents SDK with Native MCP https://ift.tt/H4Kz5Vi

Show HN: AgentKit – JavaScript Alternative to OpenAI Agents SDK with Native MCP Hi HN! I’m Tony, co-founder of Inngest. I wanted to share AgentKit, our Typescript multi-agent library we’ve been cooking and testing with some early users in prod for months. Although OpenAI’s Agents SDK has been launched since, we think an Agent framework should offer more deterministic and flexible routing, work with multiple model providers, embrace MCP (for rich tooling), and support the unstoppable and growing community of TypeScript AI developers by enabling a smooth transition to production use cases. This is why we are building AgentKit, and we’re really excited about it for a few reasons: Firstly, it’s simple. We embrace KISS principles brought by Anthropic and HuggingFace by allowing you to gradually add autonomy to your AgentKit program using primitives: - Agents: LLM calls that can be combined with prompts, tools, and MCP native support. - Networks: a simple way to get Agents to collaborate wit...
Read more