Observability · Project

Contributing

The library is MIT licensed and the most useful contribution is a new framework integration. That work is deliberately mechanical: find the seam, write the mapping onto one documented event model, add an example. The rules below are not style preferences — each exists because breaking it turns an observability package into an outage.

Source: Cognipeer/cognipeer-observability · Issues

Layout

js/       @cognipeer/observability      TypeScript package (plus the cognipeer-n8n CLI)
python/   cognipeer-observability       Python package
examples/ runnable examples per framework
docs/     the ingest contract every integration maps onto

The two packages are independent — no shared build, no code generation. They are kept behaviourally in step by mirroring each other's structure and by sharing one specification, the data model.

Local setup

# TypeScript
cd js && npm install && npm run build && npm test

# Python
cd python && pip install -e ".[all,dev]" && pytest

The rules an integration must follow

1. Never raise into the traced application. Wrap your handler bodies. Most frameworks swallow exporter exceptions and log at warn level, so a crashing integration is invisible and simply looks like "tracing doesn't work".
2. Never block the traced application. Do not await network I/O in a callback. Record into the session; the transport delivers on its own thread (Python) or promise chain (JavaScript).
3. Import the framework lazily. It is an optional peer dependency in JavaScript and an extra in Python. Importing the core must never pull it in.
4. Absent, not zero. If the framework did not report token usage, leave the field unset. A zero silently under-reports spend.
5. Say what you cannot capture. Every integration's guide carries a "what is captured" table with honest negative entries and a one-line reason. A framework that cannot expose tool schemas is a fact to document, not to paper over.
6. Do not invent data. Where a value is reconstructed rather than observed — as n8n's tool menu is, from the workflow JSON — mark its provenance in metadata.

Adding a framework

1. Find the seam. In order of preference: a first-class tracing or callback interface, then an event bus, then an OpenTelemetry instrumentor already emitting OpenInference or OpenLLMetry attributes (in which case the existing OTLP exporter may already cover it and no new code is needed), then middleware, then monkeypatching as a last resort — and say so in the docs if it comes to that.
2. Read the framework's source, not just its docs. This ecosystem moves fast enough that published docs are routinely a version or two behind, and argument order has been known to differ between a package's .d.ts and its runtime.
3. Write the mapping onto the data model. Use openSpan / closeSpan (JavaScript) or open_span / close_span (Python) when the framework gives you paired start and end callbacks — one pair becomes one event carrying both sides. The full signatures are in the API reference.
4. Register the entry point. JavaScript: an exports subpath in package.json and an entry in tsup.config.ts. Python: a top-level module plus an extra in pyproject.toml.
5. Add a runnable example under examples/, following the shape of the existing ones.
6. Add the doc page, following the shape of the shipped integration guides: orientation, minimal wiring, options table, thread grouping, what is captured, version matrix, gotchas.

Testing

Unit-test the mapping, not the network. Both packages let you swap the transport, so an integration test is: drive the real framework with a fake model, capture what would have gone on the wire, and assert on the payload.

from cognipeer_observability import _transport

sent = []
_transport.Transport._submit = lambda self, path, payload: sent.append((path, payload))

A test that needs a live model API key does not belong in CI.

Pull requests

Keep them to one framework. Include the framework versions you tested against — "works on LangChain" is not checkable, "verified on langchain-core 1.2.8 and 0.3.75" is.

Roadmap

Two frameworks currently arrive through the OpenTelemetry route but have a first-class seam that would capture strictly more:

• CrewAI — its public event bus (crewai.events: BaseEventListener, crewai_event_bus) reports token usage and tool lifecycle natively, where the OTel instrumentors resort to monkeypatching.
• LlamaIndex — get_dispatcher() with BaseEventHandler and BaseSpanHandler exposes real tool definitions on LLMChatStartEvent.additional_kwargs["tools"], which the OTel path does not reliably carry.

Also wanted: Mastra, whose ObservabilityExporter provides real 32-hex trace ids and a first-class conversation id, and C# / .NET, where Semantic Kernel and the Microsoft Agent Framework are OpenTelemetry-native — so an OTLP endpoint works today and a native package would be a convenience rather than a necessity.

Licence

MIT. Cognipeer Console itself is licensed separately — see Licensing.