On the Geometry of Concept Representation in Language Models
We show that high-level concepts in transformer models form predictable geometric structures, enabling targeted concept editing without retraining.
Olyxee Lab is our research division — dedicated to advancing the science of artificial intelligence. We work on fundamental problems in AI safety, interpretability, and capability, and publish our findings openly.
Pushing the boundaries of what AI systems can understand, reason about, and generate. We explore novel architectures and training paradigms.
Developing methods to ensure AI systems behave as intended. Mechanistic interpretability, reward modeling, and formal verification of AI behavior.
Understanding how neural networks represent and process information internally. Making the black box transparent without sacrificing capability.
Building AI systems that achieve more with less — smaller models, fewer parameters, lower energy. Intelligence shouldn't require a data center.
Systems that can perceive, reason across, and generate content spanning text, vision, audio, and structured data simultaneously.
Studying how complex behaviors arise from simple training objectives. Understanding and predicting capability jumps in scaled systems.
We show that high-level concepts in transformer models form predictable geometric structures, enabling targeted concept editing without retraining.
A framework for systematic pre-deployment verification of AI models targeting heterogeneous edge hardware.
We demonstrate that structured sparse attention patterns can match dense attention quality at 40% of the compute budget.
A comprehensive classification of how AI systems fail in production, with detection signatures and mitigation strategies for each mode.
We publish our findings, share our methods, and contribute to the broader scientific community. Knowledge compounds when shared.
Every research direction is evaluated through the lens of safety. We don't build capabilities without understanding their implications.
Our best research becomes Olyxee products. The path from paper to production is short and deliberate.
We're looking for researchers and engineers who want to work on problems that matter.