AI Systems Evaluation & Reliability Internship

Cognisos

Location: Remote
Workplace: Remote
Employment: Internship
Salary: —

Apply on the employer’s site

Posted 1mo ago

AI Systems Evaluation & Reliability Internship at cognisos

cognisos AI Systems Evaluation & Reliability Internship Remote

Intern Apply for AI Systems Evaluation & Reliability Internship

Support in validating the core processing pipeline alongside our founding team

About cognisos

Cognisos is a research lab that is actively building and shipping products. Our first product is Fabric, an evolving memory layer for agents. Through Fabric, agents reduce hallucinations, improve context windows and deliver higher quality. The final result is we unlock quality, trustworthy workflows to build at scale faster than any alternative. We are a small, yet powerful team today, blending expertise in different domains to drive everything we do. Come join us!

Description

Role

We are looking for an engineering intern to help us over the summer in bringing our product to market. The primary role needed is around validating the core processing pipeline throug: benchmarks, regression tests, PR QA, failure analysis, and reproducible reports.

This is primarily an engineering role. Product/backend work is Rust-first, while benchmark and evaluation workflows are more Python-heavy. The ideal candidate is systems-minded, comfortable learning Rust, and effective with Python for scripting and analysis.

What You Will Work On

Run and analyze memory/recall benchmarks.
Add benchmark controls and ablations, such as no-context baselines, retrieval-vs-reader failure analysis, and regression checks.
QA agent-generated PRs by running tests, reading logs, checking edge cases, and validating behavior.
Write small Python, Rust, and SQL tools for benchmark analysis, smoke tests, and CI validation.
Maintain concise runbooks so benchmark and development workflows are repeatable.
Help summarize results clearly for the team, including what passed, what failed, and what the result actually means.

Our Ideal Candidate

Strong CS fundamentals and fast learning ability.
Comfortable with terminal workflows, Git/GitHub, and CI logs.
Python for benchmark/eval scripting.
Rust interest or experience, since the product codebase is Rust-first.
Basic SQL/Postgres familiarity.
Good testing and debugging instincts.
Familiarity with or strong interest in LLM/RAG concepts: embeddings, vector search, evals, model-as-judge, context windows.
Clear written communication and high attention to detail.
High agency: able to take an ambiguous validation task and make steady progress.

Nice To Have

Rust project experience.
Docker or CI experience.
Prior ML/AI project.
Basic statistics: accuracy, confidence intervals, sampling, ablations.
Ideal Project For July-August
Help build the validation layer around Fabric: benchmark runbooks, no-context controls, regression smoke tests, PR QA checklists, failure analysis reports, and lightweight tools that let us ship agent-generated code faster without losing correctness.

Ideal Profile

Someone careful, technical, and high-agency who can help turn fast agent-driven development into trusted, benchmarked, shippable work.

Apply for AI Systems Evaluation & Reliability Internship

Your Privacy Choices Read our Privacy Policy

By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts.

Cookies settings Reject all Accept all

Privacy Preference Center

When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.

Learn more information about your privacy

Allow all

Manage consent preferences

Strictly Necessary Cookies

Always active

These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.

Functional Cookies

These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.

Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Cookie List

Clear checkbox label label

Apply Cancel

Consent Leg.Interest

checkbox label label

Reject all Confirm my choices

Skills

Python
Rust
SQL
Postgres
Git
GitHub
Docker