tboudreaux/cex
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
Emily Boudreaux
2026-09-07 15:16:52 -04:00
commit e8ce633e5e
54 changed files with 13079 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

201
README.md Normal file
View File

@@ -0,0 +1,201 @@
# CEX — Code Explorer
A read-only desktop code browser for Linux and macOS. Milestones 1–3 provide workspace setup, C/C++ highlighting and folding, file browsing, text search with saved exclusions, and clangd-backed definitions, references, callers, type information, and diagnostics, plus configurable LLM explanations with reviewed source context. Exhaustive compile-time analysis remains a future optional backend.
## Run
Install a current stable Rust toolchain and Node.js 22 or newer, then install the [Tauri platform prerequisites](https://tauri.app/start/prerequisites/).
On Fedora:
```sh
sudo dnf install gtk3-devel webkit2gtk4.1-devel librsvg2-devel patchelf
```
On macOS, install the Xcode command-line tools (`xcode-select --install`).
From this repository:
```sh
npm ci
npm run desktop
```
The first launch compiles the Rust desktop dependencies and can take a few minutes. `npm run dev` alone serves a browser preview; project access requires the desktop app.
On Linux, CEX defaults to WebKitGTK’s compatibility rendering path (`WEBKIT_DISABLE_DMABUF_RENDERER=1`) to avoid blank windows and “Failed to create GBM buffer” errors on affected graphics drivers. This is applied inside the executable, so it also covers development launches. An explicitly supplied value is preserved; use `WEBKIT_DISABLE_DMABUF_RENDERER=0` to opt back into DMA-BUF rendering. macOS is unaffected.
To compile a standalone executable without a development server:
```sh
npm run tauri -- build --debug
```
The executable is `target/debug/cex`. Installer packaging is disabled for this milestone.
## Workspace and text search
1. Click **Open a project** and select a source directory, or type its absolute path.
2. Optionally select an existing Meson or CMake build directory. It must have `meson-info/intro-projectinfo.json` (Meson) or `CMakeCache.txt` (CMake), plus a nonempty, valid `compile_commands.json`. Omit the build directory to browse any folder. CEX never configures or builds the opened project.
3. Expand directories and open a C/C++ file. Verify highlighting, line numbers, text selection/copying, and that typing does not modify the file. Use gutter arrows to fold indented blocks. **Find in file** opens Monaco's text search.
4. Choose **Search**, or press **Ctrl/Cmd+Shift+F**, and enter literal, case-sensitive text. Click a result to jump to its highlighted location. Use **Cancel** during a long search; changing the query cancels the preceding request.
5. Close and relaunch CEX, select the same source directory, and verify that the saved build directory and last viewed file are restored. The Open Recent landing page lists up to 20 successfully opened projects, newest first. Select one to restore its saved build and last file. The × button removes only the history entry. Missing projects or invalid builds open the setup dialog for correction. Global history lives in `~/.cex/recent-projects.json`; existing projects appear after you open them once in this version.
6. Under **Workspace…**, select **Force reset**, confirm, and reopen. The last file and caches are cleared, while source/build files are preserved. The build selection shown in the dialog becomes the new configuration.
In the Search panel, enter exclusion globs (one per line), such as `docs/`, `/vendor/`, or `**/generated/**`, then click **Apply exclusions**. These settings persist in `.cex/workspace.json` and affect text search only. A bare directory pattern matches that directory name at any depth; a leading slash anchors it to the project root.
Use the refresh button above the file tree after externally adding/removing files. Reopening a file reloads its contents; this version does not watch external changes automatically.
Use the header’s **System / Light / Dark** selector to change appearance. System follows your OS preference; an explicit choice applies to the entire interface, editor, and graph, and is saved in `~/.cex/appearance.json`. The small panel icon beside it toggles the Chat/Graph sidebar.
## Try milestone 2: C/C++ navigation
Install clangd separately (Fedora package `clang-tools-extra`; on macOS use an LLVM installation with clangd). Version 22 is tested here. Set the executable name or absolute path in **Workspace…** if it is not available on PATH. Select a Meson or CMake build directory to activate navigation. Without a build or clangd, file browsing/search still works.
- Click a symbol, then **Definition** (F12 or Ctrl/Cmd-click). A single result opens directly; multiple results appear in the Navigation panel. **Back** or Alt+Left returns to your previous location.
- **References** (Shift+F12) finds uses and declarations, including uses of concepts/types. **Callers** finds incoming calls to functions and opens the actual call site, labelled with its enclosing caller. It is not an exhaustive dynamic call graph.
- **Type info** or hovering shows clangd's type/documentation information. CEX displays compiler-backed information, not an LLM explanation.
- The status button opens the Navigation panel, which reports startup/indexing/errors. Diagnostics appear as editor markers with hover messages. Folding uses clangd ranges when available, with indentation folding as the fallback.
- **Restart analysis** stops/restarts clangd and reloads the original compilation database. Use it after changing build configuration or rebuilding generated headers. Reopening a file refreshes its content. Restart also retries after a language-server failure.
CMake builds use the same clangd navigation, concept graphs, and explanation context. Configure your project with compilation database export enabled, then select the resulting build directory in **Workspace…**:
```sh
cmake -S /path/to/source -B /path/to/build -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
```
For an existing Ninja or Makefile build, keep its generator and enable export with `cmake -S /path/to/source -B /path/to/build -DCMAKE_EXPORT_COMPILE_COMMANDS=ON`. [CMake supports this export with Ninja and Makefile generators](https://cmake.org/cmake/help/latest/variable/CMAKE_EXPORT_COMPILE_COMMANDS.html); Xcode does not export it. CEX never runs these commands itself. If you use presets, enable export in your configure preset or configuration command and select its resolved `binaryDir`, not the source directory or a `Debug`/`Release` output subdirectory. CEX does not evaluate presets or choose among multiple configurations; clangd uses the selected database. Separate build directories per configuration are recommended when their flags differ.
Builds may be inside the source tree or elsewhere, including paths with spaces. In-source builds (where source and build are the same directory) remain unsupported. Generated headers must already exist; CEX preserves the database's working directories and include flags when copying it into `.cex/clangd`.
A C++23 CMake example includes a generated header and cross-file navigation:
```sh
cmake -S examples/cmake -B /tmp/cex-cmake-build -G Ninja -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
```
Open `examples/cmake` and select `/tmp/cex-cmake-build`. The ignored Rust test `cmake_cpp23_navigation_with_generated_headers` configures a temporary copy with real CMake and checks definitions, callers, generated-header resolution, and unchanged build metadata.
A small C++23 Meson example is included:
```sh
meson setup /tmp/cex-navigation-build examples/navigation
```
Open `examples/navigation` as the source directory and `/tmp/cex-navigation-build` as its build directory. Open `numbers.hpp`, select `increment`, and try Definition/Callers. Select `Addable` for References. Searching for `increment` also finds `docs/notes.md`; adding `docs/` to exclusions removes that text result.
Results reflect only the selected build, and may be incomplete during indexing, with missing generated headers/parse errors, or for indirect/template-dependent calls. C++23 parsing uses the build's flags; CEX does not override the selected language standard. Search exclusions do not filter semantic references/callers.
## Try milestone 3: explain code
1. Open a C/C++ file with clangd active, click a symbol (for example `Addable` or `increment` in the example), then **Explain…**. A highlighted selection can also be explained. Context preparation stays local.
2. Open **Model settings**. Enter an OpenAI-compatible API base URL (such as `http://localhost:1234/v1` or `https://api.openai.com/v1`) and a model identifier accepted by that server. CEX appends `/chat/completions`.
3. For authenticated servers, enter a session API key or the name of an environment variable available when CEX starts. Session keys take precedence and are never saved in `.cex`; switching projects or endpoints clears them. Local servers can omit authentication.
4. Choose an audience profile: beginner, learning C++, expert, or custom. Each profile's instructions are editable and saved with the project. Configure the prompt byte budget, output token limit, and streaming. Older compatible servers may need `max_tokens` instead of `max_completion_tokens`, or no output-limit parameter.
5. Optionally enter a question and click **Prepare context**. Inspect the destination, source snapshots, omissions, and exact request messages. **Send explanation** sends that reviewed snapshot to the configured endpoint. Editing the question updates the request using the existing snapshots; no source collection is needed. Changing model settings requires preparation again.
6. Under **Called-function bodies**, select individual direct callees or **Select all**, choose the recursion depth, and click **Apply body selection**. Depth **0** adds no bodies, **1** adds selected direct callees, and **2** also adds their callees. Two is the maximum, enforced in the backend. Definitions are resolved across files (using clangd’s explicit definition location from [symbol information](https://github.com/llvm/llvm-project/blob/main/clang-tools-extra/clangd/Protocol.h) when available); shared/recursive calls are deduplicated. **Select none** removes the added bodies. These choices apply to the current prepared selection.
7. Review the updated byte count and warnings before sending. Large contexts (32 KiB or at least 75% of the configured budget) show a latency/cost warning. Traversal considers at most 80 functions, and the existing 12-file, 16-snapshot, per-snippet and total prompt limits still apply; selected bodies may therefore be omitted or truncated. Clangd cannot always resolve indirect calls or find library bodies. Inspect the actual snapshots to see what will be sent.
8. The response streams into the panel. Click a source citation such as **[S1]** to open its reviewed location. Unknown citations are labelled; model-provided URLs are not actionable. **Cancel** stops the request and labels any partial output. Requests can be retried with the same reviewed snapshot.
Right-click a symbol in the editor for **Go to Definition**, **Find References**, **Find Callers**, **Show Type Information**, and **Explain Selected Code…**. A right-click inside highlighted code preserves the selection for explaining.
Context includes the enclosing declaration or selected lines, clangd type information, definition locations, and structured references/callers/callees with bounded surrounding source. Concepts use references; function call hierarchy information is included where clangd supports it. Missing context and truncation are explicit. Collection is capped at 12 files, 16 snapshots, 160 lines/12 KiB per snapshot, and the configured total prompt byte budget (64 KiB by default). Bytes are not model tokens: adjust the budget to your server's context window. Generated/external headers returned by clangd can be included, so review the source list before sending.
Explanations use only the selected build's ordinary clangd results. They do not enumerate all template instantiations or evaluate every compile-time path. Indexing, parse errors, and indirect calls can limit evidence. The model cannot browse files or run commands. Source/comment text is presented as untrusted evidence; generated explanations can still be wrong. Snapshots and responses are kept in memory, with no saved chat history. Preparation may wait for an in-flight clangd request when cancelled; HTTP cancellation interrupts a running model request.
The provider interface is separate from context assembly. This implementation uses the [OpenAI Chat Completions API](https://developers.openai.com/api/reference/resources/chat) for broad local-server compatibility, including optional bearer authentication, streaming SSE, and nonstream responses. No particular model or hosted provider is required.
## Composed concepts
Explaining a concept exposes **Concept dependencies** in the context panel. CEX uses [clangd’s AST](https://clangd.llvm.org/extensions#ast) to find referenced concepts in constraint expressions, including both branches of `||`, conjunctions, nested requirements, and return-type constraints such as `{ expression } -> Concept`. Definitions are resolved through clangd across files; this does not evaluate all possible template instantiations or claim which branch holds for every type.
Select individual concepts or **Select all**, then **Apply concept selection**. Depth 1 includes their definitions; depth 2 also includes the concepts those definitions reference. The same size warnings, truncation reporting, and hard limits used for functions apply. Unresolvable references and unavailable AST support are reported in the context omissions.
**Include std namespace concepts** is off by default, applying to both direct and nested dependencies. Standard concepts remain visible in the list but cannot be selected until you enable the option. Namespace classification uses clangd’s resolved declaration namespace, so namespace aliases and `using std::...` are handled. Selecting a standard concept itself for explanation still includes that selected source; the switch controls added dependency definitions. This option applies to the current prepared context and resets to off for a new preparation.
A test project includes `Arithmetic`, `Sized`, `Flexible`, and `Outer`, with dependencies split across headers:
```sh
meson setup /tmp/cex-concepts-build examples/concepts
```
Open `examples/concepts` with that build directory, select `Flexible` in `composed.hpp`, and compare depths 1 and 2. `Addable` appears at depth 2 through `Arithmetic`. Enable standard concepts to include `std::copyable` as well. `main.cpp` contains static assertions covering integer, floating-point, class, and rejected types.
## Concept graph sidebar
The right sidebar now has **Chat** and **Graph** tabs. The **Sidebar** header button and the sidebar’s × button collapse/reopen the whole panel, preserving both views. Switching tabs preserves the prepared chat context and question.
Select a concept name and click **Graph** in the editor toolbar, or right-click **Show Concept Graph**. The graph places contributing concepts, `requires` blocks, and other constraint expressions above the selected concept. `&&` and `||` appear as explicit grouping nodes. Below it are concepts that depend on it, constrained declarations, and ordinary uses such as `static_assert`; edge labels distinguish **contributes to**, **constrains**, and **used by**.
Select a graph node to inspect its source preview and **Open source**. Concept nodes also offer **Focus this concept** to rebuild the graph around that concept. **Graph selected concept** uses the current editor cursor. **Fit** shows the whole graph; +/− zoom and scrolling let you explore larger graphs. The view uses Dagre for layout and clangd AST/reference information for its content. No LLM request is made.
This first graph view supports concepts only. Standard concepts are visible as leaves, with no automatic recursive library expansion. Results reflect the active build and may be incomplete during indexing or with parse errors. Contributions are capped at 100 nodes/30 expression levels (80 referenced concepts per requirement), usage discovery at 80 references/40 declarations/12 files, and source previews at 4,000 characters. An enclosing template declaration up to 20 lines after a constraint reference can be recovered through the AST; references that cannot be classified remain ordinary source locations. Limits and coverage notes appear below the graph.
Try `Flexible` in `examples/concepts/composed.hpp`: its arithmetic/sized alternatives and `requires` expression appear above it; `Outer`, `constrained_identity`, and ordinary checks appear below it.
## Vim reading keys
Vim reading bindings are active when the source editor has focus. Source remains read-only: there is no insert mode or editing operator. Search boxes, model prompts, and settings retain normal typing. Mouse selection/copy and existing editor shortcuts still work.
| Keys | Action |
| --- | --- |
| `h j k l`, `w b e`, `ge` | Character/line and word movement |
| `0`, `^`, `$` | Line start, first nonblank, line end |
| `gg`, `G`, `42gg` / `42G` | First line, last line, specified line |
| `Ctrl+d/u`, `Ctrl+f/b` | Half-page / full-page movement |
| `%`, `zz` | Matching bracket, center cursor |
| `gd` | Go to definition |
| `gc`, `gr` | Find callers / references and open the first result |
| `n`, `N` | Next / previous result, wrapping through the current list |
| `/` | Open in-file search; Enter or Escape returns to the editor |
| `Ctrl+o`, `K` | Back, type information |
| `za`, `zc`, `zo`, `zM`, `zR` | Toggle, close, open, close all, open all folds |
| Escape | Clear a pending chord/count or selection |
Counts such as `10j` and `3n` are supported (capped at 1,000). After `gc`/`gr`/`gd`, `n` and `N` cycle navigation results across files. After `/`, they cycle in-file search matches. The footer shows the active target. These are reading bindings built on Monaco, rather than a complete Vim emulation.
## Workspace storage and boundaries
Project settings live under `<source>/.cex/`:
```text
.cex/
owner # identifies CEX-owned storage
workspace.json # build, last file, clangd, search exclusions, nonsecret LLM settings
clangd/
compile_commands.json # local copy; original working directories retained
.cache/clangd/index/ # compiler-backed index
clangd.log # stderr, replaced on each restart
cache/ # fallback cache location
tmp/ # compiler temporary files
```
CEX checks existing state before opening. Corrupt or unsupported CEX settings can be force-reset. An unrecognized `.cex` directory or a `.cex` symlink is never overwritten/reset. Force reset deletes the owned `.cex` directory, including the compiler index, and creates fresh settings. Do not store personal files inside it. CEX does not edit your `.gitignore`; you may add `.cex/` yourself. The application WebView runs in incognito mode to avoid a persistent browser profile outside the workspace.
Browsing/search excludes `.cex`, `.git`, symlinks, and the selected build directory. The tree displays other files, including gitignored files; project search respects repository ignore rules. Search is literal and case-sensitive, returns at most 1,000 matches, and reports truncation/skipped files. Viewing and searching require UTF-8 text and currently limit each file to 4 MiB. Paths with non-UTF-8 names are skipped. C/C++ files receive syntax coloring; other text is shown as plain text. Navigation may open external or generated headers returned by clangd, even though those directories are excluded from the file tree. Arbitrary external paths are not exposed through the read command.
CEX limits clangd to at most four background workers and eight open documents. Before workspace switches, resets, and normal application exit it stops and reaps the language-server process. Its compilation database copy routes index shards into `.cex`; the original source/build files are unchanged. Project/user `.clangd` YAML configuration is currently disabled so it cannot redirect indexes or enable remote indexing. Custom compiler-driver querying is not enabled; unusual toolchains may require explicit include/target flags in the build database.
## Development and verification
```sh
cargo test -p cex-core
cargo test -p cex-core -- --ignored --nocapture # real clangd integration tests
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
npm run build
npx playwright install chromium
npm run test:ui
```
The Rust tests exercise actual temporary files: persistence/reset, malformed state, symlink boundaries, Meson/CMake validation, exclusions, Unicode search locations, cancellation, binary detection, and result/file limits. Playwright runs the actual Monaco frontend against mocked desktop IPC; it checks workspace restoration, read-only viewing, file navigation, find/search, and reset. Additional UI tests cover semantic result navigation, back, saved exclusions, and missing-clangd recovery. Real-clangd tests check C++23 definitions, concept references, callers across unopened files, and cache placement. A generated 120,000-line/120-translation-unit fixture checks all callers can be indexed; its simple code is not a performance guarantee for template-heavy projects. Local mock HTTP tests additionally cover exact reviewed payloads, authentication, fragmented UTF-8 streaming, legacy output limits, cancellation, and incomplete/error responses. UI tests cover context review, sanitized citations, custom profiles, and stale-response cancellation. No hosted model credentials are needed for these checks. Browser tests do not replace native WebKit/macOS testing.
## Structure and subsequent milestones
- `crates/cex-core`: workspace and read-only filesystem/search operations, independently testable without GUI libraries.
- `src-tauri`: asynchronous desktop commands and workspace service lifecycle.
- `frontend`: TypeScript UI and Monaco integration. Rendering uses text nodes for project content.
The frontend only receives structured results through Rust commands. Filesystem traversal, reads, and search run on blocking workers outside the UI thread; directories load lazily and superseded searches are cancelled. One editor model is retained at a time, with in-memory view positions for previously opened files.
`crates/cex-core/src/analysis` implements the `CodeAnalysis` interface using clangd over LSP, independently of text search. Its structured results identify source locations, active build configuration, analysis method, and coverage limitations. A future deeper compiler pass can implement a separate backend without replacing normal navigation. `crates/cex-core/src/explanation` consumes those results, assembles bounded evidence, and exposes an independent `ModelProvider` interface. Source uploads occur only when the user clicks **Send explanation**.