
Closed
Posted
Paid on delivery
# AI Specialist – Terminal-Bench & Outlier Projects **Position:** AI Specialist / AI Evaluation Engineer **Work Type:** Remote, Contract **Project Focus:** Terminal-Bench, AI agent evaluation, and coding task benchmarks ## About the Role We are looking for an experienced AI specialist with practical experience working on Outlier AI projects, particularly Terminal-Bench and terminal-based coding agent evaluations. The ideal candidate understands how to evaluate AI agents on complex software engineering tasks in command-line environments, analyze model behavior, troubleshoot execution failures, and produce accurate, reproducible evaluation results. This role is best suited to someone who has already worked on similar AI training, coding evaluation, or agent benchmarking projects and can demonstrate strong technical judgment. ## Responsibilities - Perform technical evaluations of AI-generated solutions in terminal-based environments. - Work with software development tools, command-line interfaces, Git, Linux, and automated test suites. - Analyze AI agent performance on coding, debugging, system administration, and software engineering tasks. - Investigate failed executions, incorrect outputs, environment configuration issues, and test failures. - Review task instructions, expected outcomes, evaluation criteria, and test reliability. - Produce clear, evidence-based assessments of AI-generated work. - Maintain reproducible testing procedures and document technical findings. - Collaborate remotely with project leads and other AI evaluation specialists. ## Required Qualifications - Demonstrable experience with Outlier AI projects, especially Terminal-Bench or closely related coding-agent evaluation work. - Strong command-line and terminal troubleshooting skills. - Proficiency in Linux, Git, scripting, and software debugging. - Experience with at least one programming language, such as Python, JavaScript, Java, or C++. - Ability to read code, execute tests, diagnose failures, and distinguish genuine solution errors from environment issues. - Strong analytical thinking and attention to detail. - Excellent written English and the ability to explain technical findings clearly. - Ability to work independently in a remote, project-based environment. ## Preferred Qualifications - Experience evaluating AI coding agents and autonomous terminal agents. - Familiarity with benchmark-driven evaluation, automated grading, test harnesses, and containerized environments such as Docker. - Experience with software engineering benchmarks, reproducibility, and task verification. - Familiarity with Python-based evaluation infrastructure and CI/CD workflows. - Experience with other AI training or evaluation platforms. ## Application Requirements Please include the following in your application: 1. A brief summary of your experience with Outlier AI projects. 2. Specific experience with Terminal-Bench or similar terminal-based AI evaluation benchmarks. 3. Your technical background, including programming languages and development environments. 4. Examples of relevant evaluation, debugging, or benchmark work, without disclosing confidential project information. 5. Your availability and preferred contract arrangement. **Important:** We are prioritizing candidates with verifiable, hands-on experience rather than general familiarity with AI tools. ## How to Apply Submit your technical background and a short description of your most relevant AI evaluation experience. Qualified candidates may be invited to complete a practical technical assessment.
Project ID: 40789762
101 proposals
Remote project
Active 4 days ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
101 freelancers are bidding on average $131 USD for this job

Hi, I am interested in contributing to your AI evaluation projects through systematic testing, debugging, technical analysis, and accurate documentation. I have a PhD in Cyber Security and a BSc in Computer Science, providing a strong foundation in computer systems, security analysis, and technical problem-solving. I understand the importance of reproducible testing, reliable evaluation criteria, and distinguishing genuine coding errors from environment-related failures. I can approach tasks methodically, examine expected versus actual outputs, investigate failures, and document evidence-based findings clearly. I am prepared to discuss my experience with AI evaluation platforms, Terminal-Bench or similar benchmarks, programming languages, Linux, Git, and command-line environments. I am also willing to complete a practical assessment to demonstrate my technical capabilities. I prioritize accuracy, analytical rigor, confidentiality, and dependable delivery in remote project environments. Kindly consider me, thank you.
$140 USD in 7 days
5.9
5.9

Hello, I am interested in supporting your Terminal-Bench and AI agent evaluation work. I can help assess terminal-based coding tasks through careful Linux troubleshooting, command-line verification, test execution, and evidence-based analysis of agent behavior. My approach focuses on separating genuine solution defects from environment, configuration, dependency, or test-harness issues while keeping every result reproducible and clearly documented. I can contribute across System Admin scenarios, debugging workflows, Git-based tasks, automated testing, and JavaScript projects. For each evaluation, I would review the instructions and expected outcomes, execute the solution in the specified environment, investigate failures methodically, and provide concise findings that project leads can confidently use. I also understand the importance of consistency when comparing agent performance across benchmark tasks. While respecting confidentiality, I would be glad to summarize comparable evaluation, debugging, or terminal-based benchmark experience, along with my availability and preferred contract arrangement. I am also comfortable completing a practical technical assessment to demonstrate my judgment and workflow. Could you clarify which Terminal-Bench task categories, evaluation harnesses, and weekly workload you expect this specialist to handle? Thank you for considering my application. I look forward to discussing how I can help produce accurate, reliable Terminal-Bench evalu
$155 USD in 4 days
5.1
5.1

My background is in CAD/CAM rather than AI evaluation, so I want to be transparent that I cannot claim hands-on Outlier AI, Terminal-Bench, Linux, Git, or coding-agent benchmark experience. I am a motivated, results-driven designer with more than 6 years of experience working carefully on technical and visual projects, including CAD/CAM, architecture, 3D, websites, graphics, logos, and Illustrator work. This has developed my attention to detail, ability to follow precise specifications, and commitment to delivering accurate results. However, the role’s core requirements are strongly focused on terminal-based software engineering evaluation, programming, debugging, and reproducible test analysis, which are not represented in my profile. If you are considering candidates for CAD/CAM, technical design, 3D, architectural, or graphic work, I would be glad to contribute with dedication and a client-focused attitude.
$30 USD in 1 day
5.6
5.6

Hi, terminal-based AI agent evaluation suits my background: 17+ years as a Linux/Windows systems engineer plus AI automation work. I am comfortable with Git, shell, Docker and automated test suites, and can tell real agent mistakes apart from environment or test reliability problems, then document findings so they can be reproduced. I have not worked on Outlier's Terminal-Bench itself, so I am happy to prove the quality on a sample task first. Question: how many tasks per week do you expect? Best regards, Khuda Dino
$140 USD in 7 days
5.4
5.4

Hi there, I see you're looking for someone with strong command-line and terminal troubleshooting skills for evaluating AI agents. My background includes extensive work in software engineering, where I often analyze code execution in UNIX-based environments and troubleshoot various software engineering tasks using Git and Linux. I've had hands-on experience with Outlier AI projects and am familiar with evaluating AI coding agents along with using containerized environments like Docker if needed. My problem-solving skills allow me to pinpoint execution failures effectively, ensuring reproducible evaluation results. I’d be happy to discuss how my experience aligns with your needs further. Regards, Ali
$90 USD in 1 day
4.3
4.3

Hi there, I see you're looking for an AI specialist with hands-on experience in evaluating agents in terminal environments, particularly with Terminal-Bench. This is right up our alley. Our team has successfully tackled Outlier AI projects, focusing on terminal-based evaluations and benchmarks. We've worked on complex coding evaluations, troubleshooting execution failures, and ensuring reproducible results. For instance, we recently enhanced an AI agent's performance by implementing a robust testing framework that improved accuracy by 30%. To achieve your goals, we would: ▸ Perform detailed evaluations in command-line environments. ▸ Analyze and document AI agent performance meticulously. ▸ Ensure that all testing procedures are clear and reproducible. I’ll personally oversee the project, coordinating our specialists to align with your vision and ensure quality results. We’d love to hear more about your objectives and how we can help bring them to life. Regards, Stefan Clavorn Collective
$150 USD in 14 days
3.9
3.9

Hi there, I am excited to submit my proposal for the role of AI Terminal-Bench Evaluation Specialist. With extensive experience in Outlier AI projects and a strong technical background, I am well-prepared to contribute effectively to your project. I have hands-on experience with Terminal-Bench and similar terminal-based AI evaluation benchmarks, where I have successfully assessed AI agents on complex software engineering tasks. My proficiency in Linux, Git, and command-line troubleshooting, combined with expertise in programming languages such as Python and Java, allows me to navigate and evaluate AI-generated solutions efficiently. My approach involves a meticulous examination of AI agent performance, focusing on coding, debugging, and system administration tasks. I am adept at investigating execution failures, analyzing incorrect outputs, and ensuring environment configurations are optimal. By maintaining reproducible testing procedures, I ensure that all assessments are evidence-based and reliable. Collaboration is key, and I am experienced in working remotely with project leads and other specialists to achieve project objectives. My strong analytical skills and attention to detail ensure accurate and clear documentation of technical findings. I am looking forward to the opportunity to leverage my skills and experience to support your project's success. Thank you for considering my application. Best Regards,
$30 USD in 5 days
3.6
3.6

My background combines software development, technical troubleshooting, and systematic evaluation of application behavior, with experience across Python, JavaScript, PHP, Laravel, APIs, and SaaS systems. For terminal-based AI evaluation, I would focus on reproducing tasks in controlled environments, examining execution logs, running automated tests, and tracing failures to their actual causes. An important part of this work is distinguishing incorrect agent-generated code from dependency issues, environment misconfiguration, test harness problems, or inconsistent task instructions. I would document findings with reproducible steps and concrete evidence rather than relying on subjective judgments. My broader technical experience includes Laravel SaaS technical reviews, API integration troubleshooting, and complex automation workflows. I’m comfortable analyzing unfamiliar codebases, investigating unexpected behavior, and communicating technical findings clearly in English. I would welcome a practical assessment to demonstrate my debugging approach, command-line problem-solving, and ability to evaluate coding-agent outputs systematically.
$85 USD in 5 days
3.8
3.8

=====$ Hi, Aleksandar here from Serbia $===== Reliable Terminal-Bench evaluation depends on separating genuine agent failures from broken environments, flaky tests, and incorrect assumptions in the task setup. I noticed you mentioned “distinguish genuine solution errors from environment issues,” so I’d reproduce failures systematically with Linux, Git, Docker, scripts, and automated tests before assigning an evidence-based result. I would build a reproducible evaluation workflow around terminal execution, debugging, test verification, and clear technical reporting. I've built similar systems. Which benchmark environment and evaluation harness are you currently using for the first batch of tasks? Let's connect over a chat so we can discuss this in more detail..
$140 USD in 1 day
3.4
3.4

Nice to meet you ,The requirements of your project match my areas of work and skills, to introduce myself. My name is Anthony Muñoz and i am the lead engineer for DS Pro IT agency. I have worked for over 10 years as a Full-Stack and software development engineer and have successfully done multiple jobs. It will be a pleasure to work together to make your project. Feel free to discuss about the project with me, greetings.
$126 USD in 7 days
3.3
3.3

Hi, -Ankit- here, from Bahrain. What matters most in building your AI terminal-Bench evaluation with Docker is answers people can trust, at a running cost that stays predictable. I'm a senior AI developer with 8+ years of experience working with AI Automation, Python, Docker and JavaScript. For your AI terminal-Bench evaluation, the real challenge is wrong answers and prompt injection. A common issue: launching without guardrails or a test set, so nobody knows how often it is wrong. My plan: first, collect real examples and write down the expected result for each. Then, build the LLM pipeline with guardrails and logging. Finally, score it against those examples and tune it before launch. I'd start with the core assistant and its test set as phase one. The timeline is in the bid details. I've done AI work in Docker, so I check early for wrong answers and prompt injection. A few questions to better understand the scope: 1. Can the agent change data, or only read it? Write actions need a confirmation step first. 2. Score each change on a fixed set of real questions? I'd recommend it, quality becomes measurable. 3. Any users in California? CCPA needs a privacy notice and deletion path. I can start right away. Happy to go over the details here in chat and suggest the best approach. Looking forward to hearing from you, -Ankit-
$100 USD in 2 days
3.0
3.0

You are looking for an AI specialist to perform technical evaluations of AI-generated solutions in terminal-based environments. To handle Terminal-Bench and coding task benchmarks, your setup will utilize Linux, Git, command-line interfaces, and automated test suites to execute tests and diagnose failures. For analyzing AI agent performance on coding, debugging, system administration, and software engineering tasks, we will investigate failed executions, incorrect outputs, environment configuration issues, and test failures using scripting and software debugging. To review task instructions, expected outcomes, evaluation criteria, and test reliability, our process will produce clear, evidence-based assessments and maintain reproducible testing procedures. For your screening questions regarding Outlier AI projects, Terminal-Bench, technical background in Python and development environments, and availability, I will document these findings clearly through remote collaboration and independent technical judgment. On Freelancer.com I hold 4.9 stars from 153 reviews. Price: USD 120 fixed for 5 days. - Adnan
$120 USD in 5 days
2.9
2.9

Hello. "TERMINAL-BENCH AI AGENT EVALUATION" — you need reliable, reproducible assessments that distinguish genuine coding-agent failures from test, environment, or configuration issues. I’d approach each task by reviewing its requirements and expected outputs, reproducing the run in the terminal, inspecting the code and logs, and executing the relevant tests. I’d use Linux, Git, and scripting tools to isolate failures and document findings with clear evidence so results can be independently verified. I want to be transparent: I can’t claim verified Outlier AI or Terminal-Bench project experience without evidence of that background. Could you share the benchmark environment and practical assessment requirements so I can confirm the fit? I look forward to discussing the technical scope.
$220 USD in 3 days
3.1
3.1

Hi, I have a strong technical interest in AI agent evaluation, software debugging, and benchmark-driven testing. I’m comfortable working with Python, Linux command-line tools, Git, automated test suites, and troubleshooting code execution failures. My approach is to evaluate AI-generated solutions systematically, reproduce failures, distinguish implementation errors from environment issues, and document findings with clear evidence and reproducible steps. I understand the importance of following benchmark instructions precisely and maintaining consistent evaluation standards. I’m interested in discussing your Terminal-Bench evaluation workflow and completing a practical assessment to demonstrate my technical skills. I can provide further details about my relevant experience and availability during our discussion. Best regards Yasir k
$50 USD in 1 day
2.7
2.7

Hi, I'm Inam. My technical approach to AI evaluation focuses on reproducing failures, checking execution logs, running tests, and distinguishing code defects from environment or configuration issues. I value evidence-based findings, clear documentation, and repeatable results using Linux, Git, and scripting tools. I want to be transparent that I cannot claim prior Outlier AI or Terminal-Bench experience without verifiable examples. If my current background fits your requirements, I would be open to discussing a practical assessment and the expected workflow. Best regards, Inam
$107 USD in 1 day
2.2
2.2

I see you are looking for an AI specialist skilled in evaluating AI agents within terminal environments, particularly focusing on Terminal-Bench evaluations. My experience in AI technologies, combined with strong analytical and problem-solving skills, positions me well for this role. I have worked on diverse AI projects where evaluating AI performance and troubleshooting has been crucial. My technical foundation in Python and familiarity with command-line tools and environments like Linux and Git enables me to execute tests and diagnose issues efficiently. I prioritize clear documentation and reproducibility of testing procedures, ensuring reliable assessment outcomes. I would start by reviewing existing benchmarks to align evaluation criteria effectively with project goals. This method ensures accuracy in assessing AI performance. I'm looking forward to discussing this opportunity further with you.
$30 USD in 5 days
1.9
1.9

Hi! We have experience in AI integrations and software development using Python and JavaScript. Our technical background includes debugging, API testing, Git-based workflows, and validating application behavior through testing and troubleshooting. We can help evaluate AI-generated code by reproducing failures, checking test results, identifying environment issues, and documenting findings clearly. However, we would need to confirm any specific Terminal-Bench or Outlier AI project experience rather than claim experience we cannot verify. Could you confirm whether hands-on Outlier AI or Terminal-Bench experience is mandatory before applying?
$140 USD in 7 days
2.0
2.0

Hi, Evaluating terminal-based AI agents requires more than checking whether a solution works once. The key is reproducing the execution environment, running the prescribed tests, diagnosing failures, and distinguishing genuine coding errors from configuration or test-harness issues. My technical background includes PHP, Laravel SaaS technical reviews, API integrations, automation workflows, and troubleshooting across connected business systems. I approach technical investigations by isolating the failure, checking inputs and dependencies, reproducing the issue, and documenting findings with clear evidence so another engineer can verify the result. I can apply this structured approach to coding-task evaluation, debugging, test verification, and technical reporting. I would be interested in understanding your evaluation environment, benchmark instructions, and expected assessment format so I can align my work with your reproducibility and quality standards. Best Muhammad Saad
$140 USD in 7 days
1.8
1.8

Hey, the requirement that stands out is telling genuine solution errors apart from environment issues. In Terminal-Bench-style tasks, that’s where most wrong verdicts come from. A test can fail because of an unpinned dependency, a network call inside the container or a timing-sensitive check, not because the agent got it wrong. So my first step on any failure is running the task’s reference solution in a clean container. If the oracle fails too, the task is broken, not the agent. From there, I’d read the agent’s full command trace, rerun the failing step on its own, and write up the verdict with the exact commands and output, so anyone can reproduce it. Technical background: Linux, Git, Bash and Python scripting, Docker, and debugging test suites in CI. I’m happy to cover my evaluation experience in detail on a call, or through your practical assessment. Availability: I’m flexible on hours and fine with an hourly or per-task arrangement. One question so I understand the setup: is the evaluation work done on your own infrastructure, or inside the Outlier platform under the contractor’s own account? Juan Pablo
$140 USD in 7 days
3.9
3.9

-**High-Quality work, Fast delivery, and results you can TRUST**-- Building your AI terminal-Bench evaluation with Docker is doable. I'm a senior AI developer with 8+ years of experience working with AI Automation, Python, Docker and JavaScript. For your AI terminal-Bench evaluation, the real challenge is wrong answers and prompt injection. My plan: first, collect real examples. Then, build with guardrails. Finally, score and tune. I'd start with the core assistant and its test set as phase one. The timeline is in the bid details. I've done AI work in Docker, so I check early for wrong answers and prompt injection. A few questions to better understand the scope: 1. Can the agent change data, or only read it? Write actions need a confirmation step first. 2. Score each change on a fixed set of real questions? I'd recommend it, quality becomes measurable. 3. Any users in California? CCPA needs a privacy notice and deletion path. Happy to go over the details here in chat and suggest the best approach. Looking forward to hearing from you, VAN HIEU
$100 USD in 2 days
0.0
0.0

Apex, United States
Payment method verified
Member since Apr 6, 2026
$30-250 USD
$2-8 USD / hour
$250-750 USD
$250-750 USD
$250-750 USD
$10-30 USD
$100 USD
₹600-1500 INR
$750-1500 USD
$10-30 USD
$250-750 USD
₹1500-12500 INR
₹1500-12500 INR
$250-750 USD
$250-750 USD
₹37500-75000 INR
$8-15 USD / hour
₹1500-12500 INR
$3000-5000 USD
£20-250 GBP
$30-250 USD
$15-25 USD / hour
$250-750 USD
£750-1500 GBP
$30-250 AUD