You mentioned that "Everything beneath the Grace Cathedral experience is built on open source foundations." How feasible is it for a novice to build such an experience. Say I wanted one for my own house (should be MUCH less challenging)
1. Capturing the entire scene with photos or video and converting ("training") these into a 3d Gaussian splat. This is what Vincent did so well with Grace. Open source tools for training are Brush and Lichtfeld Studio.
2. Creating a user experience with the resulting capture. In the case of Grace it involved adding the peek effect, camera controls, moving cars etc. This was all done using PlayCanvas as the article describes, which is also open source.
There is still some art to doing 1 well, but with a capable LLM agent and our (PlayCanvas') awesome community on Discord, you could probably manage both.
Having spent a lot of time in those sorts of churches, feels weird to walk right up to the altar.
“Show HN: Immersive Gaussian Splat tour of grace cathedral, San Francisco”
262 points | 5 days ago | 63 comments
Grace Cathedral is a San Francisco landmark on Nob Hill.
After seeing Vincent Woo's high-quality capture of the cathedral and its surrounding streets as 3D Gaussian splats, we jumped at the chance to collaborate with him. Together, we turned his splats into a browser-based, interactive experience. You can see the finished app below:
This article explores how I built this app. Let's dive in.
The PlayCanvas Engine was the perfect runtime on which to build, thanks to its WebGPU hybrid renderer and streamed SOG format.
The exterior view renders around 3.5 million Gaussian splats at a time. Because they must be sorted back to front, moving the camera means repeatedly sorting millions of splats in real time.
Our first-generation Gaussian splat renderer used WebGL 2, which only provides vertex and fragment shaders. They are well suited to drawing the splats but not to sorting them, so a worker sorted the Gaussians on the CPU and uploaded the results to the GPU. Sorting and uploading could fall behind rendering, making artifacts visible as the camera moved.
WebGPU lets us divide that work more effectively. Compute shaders cull, project and sort the active splat set on the GPU every frame, then vertex and fragment shaders render the result. Keeping the data on the GPU avoids the transfer bottleneck and makes the pipeline dramatically faster than the worker-based WebGL 2 renderer.
The experience uses this WebGPU pipeline whenever the browser supports it and falls back to the WebGL 2 renderer automatically. To support both rendering paths, the cutout, flag animation and transition shaders each have WGSL and GLSL implementations so the experience looks the same on both backends.
The splat data is delivered as streamed SOG — our open compressed splat format, extended with chunked levels of detail that the engine fetches on demand.
The experience contains separate splat sets for the exterior and interior. The exterior combines a streamed LOD set covering everything within 400 meters with a static, low-detail set for the landscape beyond it. Each mode loads only its own set and evicts the other, so device memory holds one view at a time.
Startup is tuned for time-to-first-pixel. On load, each streamed splat set starts at a single coarse LOD, so only a small fraction of the data needs to arrive before the loading screen lifts. From there, the engine streams in detail progressively within a per-device budget: 3.5 million rendered splats on desktop and 1.4 million on mobile. Mode switches are covered by a short freeze-fade: the last frame is held and faded down while the next splat set loads its coarse LOD, then the new view fades in and refines.
Beyond streaming and per-device splat budgets, a few things keep the experience smooth on modest hardware — and cool and quiet on phones:
While orbiting outside, the Peek toggle carves away the section of wall nearest the camera so you can look inside. A thin, glowing rim outlines the cut edge.
It is built on the engine's gsplat shader chunk system, which lets a project replace the gsplatModifyVS and gsplatModifyPS chunks to modify every splat and every fragment. The cutouts themselves are just box entities in the Editor, grouped under trigger volumes. As the camera moves, the trigger closest to it eases its cutout open while the others ease closed.
The shader works in two stages to keep it cheap. A per-splat stage — a compute pass on WebGPU — tests each splat's center against the cutout boxes with a signed distance function. Splats fully inside an open box are culled outright and only those near a cut surface are flagged for further processing. The fragment stage then runs a pixel-exact version of the same test for flagged splats. It reconstructs each fragment's world position from depth, fades alpha inside the box and draws a rim that maintains a constant on-screen width at any distance. The rim color is written as an HDR value so bloom picks it up. The chunks are installed only while the exterior view is active, so walk mode pays nothing for the effect.
The same chunk also animates a flag hidden somewhere in the scene. Splats inside an authored box ride a traveling wave, with amplitude ramping up from the pole edge and each splat tilting to follow the displaced cloth surface. Can you find it? 🔍
We created the moving traffic by isolating and duplicating parked cars from the original capture in the SuperSplat Editor:
Three car splats spawn at randomized intervals and travel past the cathedral at slightly different speeds. They are disabled indoors, returning their splats to the interior's rendering budget.
The exterior orbit controller moves around a rounded-box envelope fitted to the cathedral. Horizontal and vertical motion are parameterized by arc length along the walls, corners and roof, producing uniform camera speed without discontinuities. Zoom adjusts the camera's standoff from the surface, while damped input adds momentum after release.
Camera flights use authored waypoint graphs to route around the cathedral's architecture. Inside, routing uses the collision mesh to select visible nodes; outside, it tests clearance against the cathedral's orbit envelope. A shortest-path search produces a route that is converted into a cubic spline, interpolating position, orientation, focus distance and field of view. Closing the annotation rewinds the flight to the previous camera state.
A layered soundscape follows the camera. Outside, two city loops crossfade based on altitude. Inside, cathedral ambience takes over, with footsteps playing only while you walk. Even the loading screen follows the theme with a rose window drawn in CSS.
Together, these details turn the cathedral capture into a living, explorable place.
Everything beneath the Grace Cathedral experience is built on open source foundations. The PlayCanvas Engine renders the scene in the browser, SuperSplat provides the tools to edit and optimize the captures and the open SOG format makes them compact enough to stream. These are the same technologies we develop in the open for anyone to use, inspect and improve.
Want to build an experience of your own? Start with our Gaussian splatting guide for the complete workflow from capture to delivery. You can prepare your capture in the SuperSplat Editor, then follow the Your First Splat App tutorial to choose the approach that best fits you, whether that is the Engine API, the PlayCanvas Editor, React or Web Components. 🚀
If you have questions, want to share what you build or simply want to meet other creators, join the PlayCanvas and SuperSplat community on Discord. We'd love to see you there! 👋
Experience Grace Cathedral for yourself and see what the open web can do.