Show HN: Blast – Fast, multi-threaded serving engine for web browsing AI agents https://ift.tt/QKx2hd5

Show HN: Blast – Fast, multi-threaded serving engine for web browsing AI agents Hi HN! BLAST is a high-performance serving engine for browser-augmented LLMs, designed to make deploying web-browsing AI easy, fast, and cost-manageable. The goal with BLAST is to ultimately achieve google search level latencies for tasks that currently require a lot of typing and clicking around inside a browser. We're starting off with automatic parallelism, prefix caching, budgeting (memory and LLM cost), and an OpenAI-Compatible API but have a ton of ideas in the pipe! Website & Docs: https://ift.tt/lfKXp2u https://ift.tt/ZqVyipF MIT-Licensed Open-Source: https://ift.tt/bHcz9Zs Hope some folks here find this useful! Please let me know what you think in the comments or ping me on Discord. — Caleb (PhD student @ Stanford CS) https://ift.tt/bHcz9Zs May 2, 2025 at 11:12PM

Comments

Post a Comment

Popular posts from this blog

Show HN: Agent File (.af) – An open file format for agents https://ift.tt/fzI5HcG

Show HN: Agent File (.af) – An open file format for agents Hi HN - We’re building Agent File (.af), which makes it possible to re-create the exact same agent (including memories, tools, message history, configs, etc.) across different machines. A big difference between LLMs and agents is that agents have associated state: system prompts, editable memory (personality and user information), tool configurations (code and schemas), and LLM/embedding model settings. While you can run the same LLM as someone else by downloading the weights, there’s no “representation” of agents that allows you to re-create an instance of an agent across services. Agent File (.af) is an open standard file format for serializing stateful agents. Originally designed for the Letta framework, .af is a human-readable representation of all the associated state of an agent to reproduce the exact behavior and memories. To demonstrate .af, we also made a few example agents with download links to .af: - MemGPT: An agen...
Read more

Show HN: Sort lines semantically using llm-sort https://ift.tt/7vEeHKP

Show HN: Sort lines semantically using llm-sort This is a small plugin I made for Simon Willison's llm utility. You can do things like: cat names.txt | llm sort -q "Which one of these names is best for a pet seagull?" cat books.txt | llm sort -q "Which book is more related to basic vs. advanced CS topics?" I see a lot of potential marrying LLMs with classic UNIX interfaces. https://ift.tt/9GDceOu February 11, 2025 at 08:55AM
Read more

Show HN: AgentKit – JavaScript Alternative to OpenAI Agents SDK with Native MCP https://ift.tt/H4Kz5Vi

Show HN: AgentKit – JavaScript Alternative to OpenAI Agents SDK with Native MCP Hi HN! I’m Tony, co-founder of Inngest. I wanted to share AgentKit, our Typescript multi-agent library we’ve been cooking and testing with some early users in prod for months. Although OpenAI’s Agents SDK has been launched since, we think an Agent framework should offer more deterministic and flexible routing, work with multiple model providers, embrace MCP (for rich tooling), and support the unstoppable and growing community of TypeScript AI developers by enabling a smooth transition to production use cases. This is why we are building AgentKit, and we’re really excited about it for a few reasons: Firstly, it’s simple. We embrace KISS principles brought by Anthropic and HuggingFace by allowing you to gradually add autonomy to your AgentKit program using primitives: - Agents: LLM calls that can be combined with prompts, tools, and MCP native support. - Networks: a simple way to get Agents to collaborate wit...
Read more