Protobuf has LSP support. You're welcome

Watch as I don't use protobuf because it is horrible.

....

Tada!

Buf's offering of protobuf registries and codegen SDKs for microservices seems less necessary in the LLM era.

I'm starting to question many of protobuf's advantages (perhaps not the wire format). Add to that monorepos and other fads of the 2010s given the rise of LLMs.

I used to be a big believer in this stuff, but I'm quickly having my core assumptions change out from under me.

Watch as I don't use protobuf because it is horrible.

....

Tada!

Buf's offering of protobuf registries and codegen SDKs for microservices seems less necessary in the LLM era.

I'm starting to question many of protobuf's advantages (perhaps not the wire format). Add to that monorepos and other fads of the 2010s given the rise of LLMs.

I used to be a big believer in this stuff, but I'm quickly having my core assumptions change out from under me.

The real argument shouldn't be about protocols becoming obsolete, but programming languages that are "less efficient" could eventually become obsolete in favor of highly scalable and performant languages due to LLMs when the main gap becomes knowing how the tech works at a high level, and not the syntax, why code in one language over another if you don't need to worry about messing up on syntax, only about reviewing logic for sanity and correctness against business rules as well as validating that it is stable code.

Anybody who thinks that you can just chuck unstructured data into LLM and YOLO the app is an idiot.

This works up to a point, and then it doesn't. And you're left with tons of inconsistently formatted data.

My company is built on protobufs from ground up :) We use it in the database, for remote calls, on the frontend, etc. The protobuf language is not great, but it's about the right balance between too expressive and too restricting.

And the best thing is that it's compact, compared to OpenAPI.

I feel exactly that way about REST. The assumption that LLMs make schemas obsolete misses how structured outputs actually work in production. When you have probabilistic models generating code, strict contracts become more critical, not less. It is no coincidence that several major LLM platforms rely on ConnectRPC and Protobuf for their own APIs.

Anybody who thinks that you can just chuck unstructured data into LLM and YOLO the app is an idiot.

This works up to a point, and then it doesn't. And you're left with tons of inconsistently formatted data.

And the best thing is that it's compact, compared to OpenAPI.

You're right, but the adeptness of models to spin up clients and behaviors on the fly is remarkable. They're capturing the semantics of behavior at a deeper level.

If we do strict schemas, I'd like to see less ceremony around them. Tool calls instead of brittle build steps and protocol registries.

Perhaps we need new tools for this going forward.

You're right, but the adeptness of models to spin up clients and behaviors on the fly is remarkable. They're capturing the semantics of behavior at a deeper level.

If we do strict schemas, I'd like to see less ceremony around them. Tool calls instead of brittle build steps and protocol registries.

Perhaps we need new tools for this going forward.

Hm... Maybe. In my view, an IDL is part of the input that you absolutely want humans to author or carefully review at least. In my experience, the ceremony around generating code is also performed very well by LLMs. But I do agree, there's definitely some changes that are needed to integrate Protobufs better. Some languages have built-in tooling to make it seamless, but it's definitely not universal.

Buf is proud to announce the first fully-featured, production-grade LSP server for Protobuf.

The Language Server Protocol is the standard API for integrating language support into your favorite IDE or text editor, such as VSCode, IntelliJ, or Neovim. An LSP server provides the smarts that power go to definition, code completion, finding references, and semantics-aware syntax highlighting.

Before today, Protobuf lacked the same LSP support other major programming languages enjoy. We don’t want to overstate our own work, but this is a game-changer for Protobuf development: Protobuf now has modern IDE support for the first time, all powered by the Buf CLI.

At Buf, we believe that Protobuf is the best schema language, thanks to its established ecosystem of libraries and tools. Protobuf shouldn’t just be the smart choice; it should be the easiest choice too. This is why we’re releasing the Buf LSP server, as part of our family of tools that complete the Protobuf ecosystem, including Protobuf-ES (used in Chromium!), Protovalidate, ConnectRPC, and the Buf Schema Registry. No matter your project or use-case, developing with Protobuf should be the easy, obvious choice.

Here’s how you can try it out!

Installing the Buf LSP

VSCode is one of the most popular graphical editors available, and it has stellar LSP support (LSP originated with VSCode). To use the Buf LSP, just install the Buf extension. This will automatically use your installed version of the Buf CLI (which bundles the LSP server), or install one if you need it.

Neovim is Buf engineers’ favorite editor, and the Buf LSP works great with it, too. All you need to do is install the Buf CLI, and configure Neovim to use it as an LSP server.

Make sure you’ve installed the nvim-lspconfig repository, which provides default LSP config options. Follow the instructions here.
Add lspconfig.buf_ls.setup {} to your .nvimrc .

Alternatively, you can add the following somewhere in init.lua (or another config file, if you like):

vim.lsp.config('buf-lsp', {
    cmd = { 'buf', 'lsp', 'serve' },
    filetypes = { 'proto' },
    root_markers = { 'buf.yaml', '.git' },
})

Then, you can enable it with vim.lsp.enable('buf-lsp') in your .nvimrc .

Learn more about LSP and Neovim here.

If you’re using another editor, first find out how LSP integration works, and just make sure the buf lsp serve command gets run to spawn the server.

Leveraging our toolchain

Buf already maintains a Protobuf compiler frontend (the part that parses files and calls plugins, like protoc), which we call protocompile. Our implementation isn’t just fully-compliant, but it is much faster and more flexible than protoc. Our compiler is so good that even Google is using it alongside protoc in some parts of its massive codebase.

To build the Buf LSP, we’ve taken protocompile to the next level. We’ve developed a brand-new, query-driven frontend which enables incremental compilation and much better diagnostics. For example, unlike protoc, we correctly diagnose a duplicate repeated modifier:

error: encountered more than one type modifier
  --> testdata/parser/type/repeated.proto:23:14
   |
23 |     repeated repeated M x4 = 4;
   |     -------- ^^^^^^^^ help: consider removing this
   |     |
   |     first one is here

We designed a new AST and intermediate representation for precisely diagnosing these errors, instead of using FileDescriptorProto. Its flexibility makes it easy for us to implement new language features as they are added to Protobuf, such as upcoming features in Editions 2024. It is also memory-efficient, so it can handle workspaces that include very large Buf modules.

Our commitment

Our work on the Buf LSP is not done. We are always improving our diagnostics, and we’re planning to add more features to the LSP server, too:

Adding imports as an automatic fix.
Tighter integration with buf.yaml (such as automatically importing modules).
Code completion and reference lookup for custom options.
Automatic suggestion of field/enum numbers.
Dedicated Protovalidate support, including syntax highlighting for CEL snippets.

At Buf, we will always push the boundary of what you can accomplish with Protobuf, enabling schema-driven development for all!

If you want help enforcing standards on your organization’s data, contact us to see if Buf can help. If building excellent developer tooling excites you, check out our careers page!

Hacker Times