Show HN: Selfhostllm.org – Plan GPU capacity for self-hosting LLMs https://ift.tt/bRxTIjO

Show HN: Selfhostllm.org – Plan GPU capacity for self-hosting LLMs A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results. https://ift.tt/wq0Ih43 August 8, 2025 at 11:19PM

Comments

Post a Comment

Popular posts from this blog

Show HN: Agent File (.af) – An open file format for agents https://ift.tt/fzI5HcG

Show HN: Agent File (.af) – An open file format for agents Hi HN - We’re building Agent File (.af), which makes it possible to re-create the exact same agent (including memories, tools, message history, configs, etc.) across different machines. A big difference between LLMs and agents is that agents have associated state: system prompts, editable memory (personality and user information), tool configurations (code and schemas), and LLM/embedding model settings. While you can run the same LLM as someone else by downloading the weights, there’s no “representation” of agents that allows you to re-create an instance of an agent across services. Agent File (.af) is an open standard file format for serializing stateful agents. Originally designed for the Letta framework, .af is a human-readable representation of all the associated state of an agent to reproduce the exact behavior and memories. To demonstrate .af, we also made a few example agents with download links to .af: - MemGPT: An agen...
Read more

Show HN: Attach Gateway – one-command OIDC/DID auth for local LLMs https://ift.tt/crSdltX

Show HN: Attach Gateway – one-command OIDC/DID auth for local LLMs We’ve been building local and on-prem agent workflows for open-source LLMs. Engines like Ollama or vLLM ship with no auth, so every team ends up writing the same JWT proxy. Attach Gateway is a single process that sits in front of any model server and handles the boring bits: - verifies OIDC / DID JWTs - adds X-Attach-User and X-Attach-Session headers so downstream agents share the same identity - optional /a2a/tasks/send endpoint for Google-style A2A hand-offs - mirrors prompts + completions to Weaviate (runs in Docker) One `pip install attach-dev`, export a token, run `attach-gateway`, and your local Ollama is protected in under 60 seconds. Repo: https://ift.tt/fxpmaiE PyPI: https://ift.tt/N9Jhatm Would love feedback – especially from teams doing multi-agent or on-prem work. https://ift.tt/fxpmaiE June 30, 2025 at 11:38PM
Read more

Show HN: Sort lines semantically using llm-sort https://ift.tt/7vEeHKP

Show HN: Sort lines semantically using llm-sort This is a small plugin I made for Simon Willison's llm utility. You can do things like: cat names.txt | llm sort -q "Which one of these names is best for a pet seagull?" cat books.txt | llm sort -q "Which book is more related to basic vs. advanced CS topics?" I see a lot of potential marrying LLMs with classic UNIX interfaces. https://ift.tt/9GDceOu February 11, 2025 at 08:55AM
Read more