vvlad/infrence-fabric
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
No description
6 commits 1 branch 0 tags 221 KiB
  • Rust 100%
Find a file
Exact
Vlad Verestiuc 558c842620
 Add V1 wire contracts and docs 
- Introduce V1 information flows diagram and overview
- Add V1 contract docs (artifact-service-v1, gateway-api-v1,
  nats-request-v1, ipc-v1, callback-v1, membership-status-v1,
  audit-query-v1, readiness-review-v1, security-v1, observability-v1,
  audit-v1, cross-contract-v1)
- Add V1 schemas under schemas/v1 (artifact.fbs, callback.fbs,
  common.fbs, envelope.fbs, ipc.fbs, membership.fbs, nats.fbs)
- Add new docs for audit query, configuration, and readiness review
- Update ADR-011 title to “Use client JWTs at the Gateway and mTLS
  internally”
- Update architecture and component docs to reflect V1 contracts and TLS
  changes
2026-07-15 20:11:32 +03:00
crates Initialize cargo workspace for fabric project 2026-07-13 22:44:01 +03:00
docs Add V1 wire contracts and docs 2026-07-15 20:11:32 +03:00
schemas/v1 Add V1 wire contracts and docs 2026-07-15 20:11:32 +03:00
.gitignore Initialize cargo workspace for fabric project 2026-07-13 22:44:01 +03:00
Cargo.lock Initialize cargo workspace for fabric project 2026-07-13 22:44:01 +03:00
Cargo.toml Initialize cargo workspace for fabric project 2026-07-13 22:44:01 +03:00
README.md Switch IPC from memfd to Unix-domain sockets 2026-07-15 13:45:06 +03:00

README.md

fabric

fabric is a distributed inference engine for on-premise and private inference clusters.

Rust services handle the infrastructure around inference: API access, request routing, artifacts, runtime supervision, response delivery, observability, and auditing. Isolated Python runtimes focus only on loading model variants and executing inference.

Note

The repository is currently an initial Rust workspace skeleton. The architecture is documented, but the described services are not yet fully implemented.

Architecture

Client
  |
  | OpenAI-compatible API
  v
API Gateway -- NATS JetStream --> Inference Node
     ^                              |
      |                              | Unix socket + FlatBuffers
     | HTTP/2 response callback     v
     +---------------------- Python Runtime

Gateway and Inference Node -- HTTP/2 --> Artifact Service

Key decisions:

  • OpenAI-compatible inference API covering text, audio, image, and multimodal endpoint families; completions and embeddings are the first milestone
  • NATS JetStream request routing through <modality>.<model>.<region> subjects
  • Isolated Python Runtime Master Process per model_name@artifact-version
  • Unix-domain sockets and FlatBuffers for Rust/Python inter-process communication
  • Direct authenticated HTTP/2 response streaming to the originating Gateway
  • Remote pluggable Artifact Service with local filesystem and S3-compatible backends
  • OpenTelemetry traces, metrics, and structured logs
  • Durable, tamper-evident audit pipeline

Workspace

Crate Intended responsibility
fabric-core Shared types, traits, and utilities
fabric-ipc Rust/Python inter-process communication
fabric-runtime Runtime orchestration, lifecycle, routing, and supervision
fabric-observability Metrics, tracing, structured logging, and auditing

Documentation

Development

Requirements:

  • Rust toolchain with Cargo
  • Linux for the target on-premise deployment environment
cargo build
cargo test
cargo fmt
cargo clippy