
Closed
Posted
Paid on delivery
# AI Specialist – Terminal-Bench & Outlier Projects **Position:** AI Specialist / AI Evaluation Engineer **Work Type:** Remote, Contract **Project Focus:** Terminal-Bench, AI agent evaluation, and coding task benchmarks ## About the Role We are looking for an experienced AI specialist with practical experience working on Outlier AI projects, particularly Terminal-Bench and terminal-based coding agent evaluations. The ideal candidate understands how to evaluate AI agents on complex software engineering tasks in command-line environments, analyze model behavior, troubleshoot execution failures, and produce accurate, reproducible evaluation results. This role is best suited to someone who has already worked on similar AI training, coding evaluation, or agent benchmarking projects and can demonstrate strong technical judgment. ## Responsibilities - Perform technical evaluations of AI-generated solutions in terminal-based environments. - Work with software development tools, command-line interfaces, Git, Linux, and automated test suites. - Analyze AI agent performance on coding, debugging, system administration, and software engineering tasks. - Investigate failed executions, incorrect outputs, environment configuration issues, and test failures. - Review task instructions, expected outcomes, evaluation criteria, and test reliability. - Produce clear, evidence-based assessments of AI-generated work. - Maintain reproducible testing procedures and document technical findings. - Collaborate remotely with project leads and other AI evaluation specialists. ## Required Qualifications - Demonstrable experience with Outlier AI projects, especially Terminal-Bench or closely related coding-agent evaluation work. - Strong command-line and terminal troubleshooting skills. - Proficiency in Linux, Git, scripting, and software debugging. - Experience with at least one programming language, such as Python, JavaScript, Java, or C++. - Ability to read code, execute tests, diagnose failures, and distinguish genuine solution errors from environment issues. - Strong analytical thinking and attention to detail. - Excellent written English and the ability to explain technical findings clearly. - Ability to work independently in a remote, project-based environment. ## Preferred Qualifications - Experience evaluating AI coding agents and autonomous terminal agents. - Familiarity with benchmark-driven evaluation, automated grading, test harnesses, and containerized environments such as Docker. - Experience with software engineering benchmarks, reproducibility, and task verification. - Familiarity with Python-based evaluation infrastructure and CI/CD workflows. - Experience with other AI training or evaluation platforms. ## Application Requirements Please include the following in your application: 1. A brief summary of your experience with Outlier AI projects. 2. Specific experience with Terminal-Bench or similar terminal-based AI evaluation benchmarks. 3. Your technical background, including programming languages and development environments. 4. Examples of relevant evaluation, debugging, or benchmark work, without disclosing confidential project information. 5. Your availability and preferred contract arrangement. **Important:** We are prioritizing candidates with verifiable, hands-on experience rather than general familiarity with AI tools. ## How to Apply Submit your technical background and a short description of your most relevant AI evaluation experience. Qualified candidates may be invited to complete a practical technical assessment.
Project ID: 40793283
122 proposals
Remote project
Active 7 hours ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
122 freelancers are bidding on average $427 USD for this job

Hello, Live Experts combines over a decade of technical delivery experience with strong Linux and computer security expertise, an effective foundation for evaluating terminal-based AI agents, troubleshooting execution failures, and distinguishing genuine solution errors from environment or configuration issues. Our team works extensively with Python, Git, Docker, AWS, Node.js, and software debugging across AI/ML, automation, embedded, and industrial systems. This practical engineering background supports careful code review, command-line testing, reproducibility, and evidence-based reporting. We can also analyze failures across complex environments while documenting findings clearly and professionally in English. We would be glad to contribute as a reliable AI evaluation specialist for Terminal-Bench and related coding-agent benchmark work. Choose Live Experts for technically grounded, detail-focused evaluations. Thanks!
$750 USD in 3 days
6.5
6.5

⭐⭐⭐⭐⭐ AI Specialist for Terminal‑Bench and Outlier Evaluation ❇️ Hi My Friend, I hope you are doing well. I checked all of your project requirements and I can see you are looking for an AI Evaluation Engineer. You have no need to look any further as Zohaib is here to help you! My team is already doing 50+ similar projects for AI agent benchmarking. I will build a reproducible test harness that runs Terminal‑Bench tasks, captures logs, and generates clear performance reports. I will also add a quick Docker setup guide in the same budget. ➡️ Why Me? I can easily do your evaluation project as I have 5 years of experience in Python, Linux, Git, Docker, AI benchmarking, and script automation. Not only this, I have a strong grip on CI/CD pipelines, test suite integration, and code debugging. ➡️ Let's have a quick chat to discuss your project in detail and let me show you samples of my previous work. Looking forward to discussing with you in chat. ➡️ Skills & Experience: ✅ Python ✅ Linux CLI ✅ Docker ✅ Git ✅ Bash Scripting ✅ AI Benchmarking ✅ Terminal‑Bench ✅ Outlier AI ✅ Test Automation ✅ CI/CD ✅ Code Debugging ✅ Technical Reporting Waiting for your response! Best Regards, Zohaib
$350 USD in 2 days
5.5
5.5

Hi, I am a Linux systems engineer and AI automation architect with 17+ years in IT, so terminal-based agent evaluation fits my daily work: Git, shell, Docker, test suites, and debugging environment and configuration failures. I can review agent solutions on coding and sysadmin tasks, separate real model errors from flaky tests or broken environments, and write clear, reproducible, evidence-based assessments. To be transparent: I have not worked on Outlier's Terminal-Bench project itself, but I can show my method on a sample task. Question: do you have a trial task I can complete first? Best regards, Khuda Dino
$500 USD in 14 days
5.4
5.4

I’d approach this like an evaluation job, not just “running tests.” Terminal-Bench work needs someone who can spot where models fail, reproduce issues cleanly, and turn that into useful signals for improvement. I can handle the full loop: 1) set up the environment and verify runs 2) execute tasks systematically and log failures 3) analyze patterns in model behavior and edge cases 4) tighten prompts/data/eval criteria so results are actionable I’m comfortable with Linux, Python, automation, and security-minded workflows, so I’ll keep the process clean, repeatable, and reliable. If you want someone who pays attention to the details and doesn’t miss the weird failure modes, I’m a good fit.
$600 USD in 2 days
4.7
4.7

Hi there, I see you are looking for an AI Specialist experienced in evaluating AI agents, particularly in terminal-based environments like Terminal-Bench. My hands-on experience with Outlier AI projects aligns with your requirements, as I have evaluated coding tasks using command-line interfaces and have a strong grasp of troubleshooting in Linux and Git environments. I am well-versed in analyzing AI agent performance, producing reproducible evaluations, and documenting findings clearly. Given my background in software engineering and my familiarity with scripting languages like Python, I can effectively assess the performance of coding agents on complex tasks. I’m available to discuss your project requirements further and am ready to deliver valuable insights. Regards, Ali
$340 USD in 10 days
4.3
4.3

Hello, AI AGENT EVALUATION & TERMINAL-BENCH {{{ I CAN SUPPORT TECHNICAL AI EVALUATION, DEBUGGING, AND BENCHMARK VALIDATION! }}} I have 12 years of experience in the IT domain, with a strong background in software development, Python, JavaScript, Linux, Git, debugging, and automated testing. I can evaluate AI-generated code, execute terminal-based tasks, investigate test failures, identify environment issues, validate outputs, and document reproducible results. I am also familiar with Docker, CI/CD workflows, and AI-powered development tools. I would like to clarify that I cannot claim prior Outlier AI or Terminal-Bench project experience without verifiable hands-on work. I am open to a practical technical assessment to demonstrate my capabilities. Thanks, Christina
$250 USD in 7 days
4.4
4.4

Hi, I have 7+ years of software development experience involving Python, C++, Linux, Git, Docker, debugging, and automated testing. I can evaluate AI-generated code, troubleshoot terminal execution failures, verify test results, and document reproducible findings. I'm familiar with terminal-based coding evaluation workflows and can demonstrate my technical skills through a practical assessment. Available for remote contract work and ready to start immediately!
$320 USD in 2 days
4.2
4.2

Hi there, I am writing to express my interest in the AI Terminal-Bench Evaluation Specialist position. With substantial experience in AI agent evaluation and coding task benchmarks, I am confident in my ability to contribute effectively to your project. Having worked extensively on Outlier AI projects, including Terminal-Bench, I have honed my skills in evaluating AI agents on complex software engineering tasks within command-line environments. My proficiency in Linux, Git, and scripting, coupled with a solid understanding of software debugging, positions me well to analyze and improve AI-generated solutions accurately. I have a strong technical background in Python and Java, enabling me to diagnose execution failures and distinguish between genuine errors and environment issues. My approach involves meticulous troubleshooting and maintaining reproducible testing procedures, ensuring reliable evaluation results. I am adept at documenting technical findings and communicating them clearly, which aligns with your project's needs for evidence-based assessments. I am enthusiastic about collaborating remotely with your team, bringing a detail-oriented and analytical mindset to the project. I am eager to apply my expertise in AI evaluation and contribute to the success of your Terminal-Bench initiatives. Thank you for considering my application. I look forward to the possibility of discussing how I can support your project further. Best Regards,
$250 USD in 10 days
3.6
3.6

===== Precision in Every Step. Value in Every Result!!! ===== Hello - Ricardo here from Buenos Aires. "TERMINAL-BENCH EVALUATION" - you need reliable AI agent testing and clear, reproducible results. I can help assess coding outputs, run tests, investigate failures, and separate real code issues from environment problems. My approach focuses on evidence, careful debugging, and clear technical reports so each evaluation is accurate and easy to verify. As we move forward, let us remember that reliable benchmarks depend on fair tests, consistent methods, and well-documented findings. Feel free to message me if you would like to discuss the details. I'd be happy to review your requirements and suggest the best next steps.
$400 USD in 5 days
3.7
3.7

Hi, They need a specialist to troubleshoot and analyze AI agent behaviors effectively, ensuring reliable evaluations. What specific AI agent behaviors are you finding most problematic during evaluation? Here's how I'd do it: - Review the current evaluation criteria and existing assessment workflows. - Set up a reproducible testing framework using Python and shell scripting in a Linux environment. - Deliver a documented evaluation of a selected AI agent's performance, highlighting clear benchmarks for success. I can have a working version of the core piece ready on a test link within a day. What coding tasks are you most interested in evaluating? Toqeer S
$430 USD in 15 days
3.1
3.1

Nice to meet you , It is a pleasure to communicate with you. My name is Anthony Muñoz, I am the lead engineer for DSPro IT agency and I would like to offer you my professional services. I have more than 10 years of working as a Backend and Software developer, I have successfully completed numerous jobs similar to yours therefore, and after carefully reading the requirements of your project, I consider this job to be suitable to my area of knowledge and skills. I would love to work together to make this project a reality. I greatly appreciate the time provided and I remain pending for any questions or comments. Feel free to contact me. Greetings
$430 USD in 7 days
3.3
3.3

My 10+ years developing and troubleshooting Java/J2EE enterprise applications provide a strong technical foundation for Terminal-Bench and coding-agent evaluation work. I can read complex code, execute and interpret automated tests, trace failures, and distinguish application defects from configuration or environment issues. My background includes Java, Spring Boot, REST APIs, Hibernate/JPA, JavaScript, AngularJS, React, Node.js, JUnit, Mockito, PostgreSQL, MySQL and MongoDB. I also work with Linux, Docker, CI/CD pipelines and AWS deployments, which aligns well with terminal-based environments, reproducible testing and containerized evaluation workflows. My experience with test-driven development and AI tools for code quality improvement and unit-test generation further supports careful assessment of AI-generated solutions. I work comfortably independently or within remote Agile teams and document technical findings clearly. I would be glad to contribute my software engineering, Linux and debugging expertise to your AI evaluation projects.
$270 USD in 7 days
3.2
3.2

Hello. "TERMINAL-BASED AI AGENT EVALUATION" — you need reliable results that separate real coding failures from environment or test issues. I’d run tasks in a controlled Linux environment, verify results against the expected behavior, and record reproducible steps for each failure. I can share relevant experience honestly, without exposing confidential project details. Is prior Outlier or Terminal-Bench experience mandatory, or will you consider candidates with demonstrable Linux debugging and benchmark-testing skills? Looking forward to working with you.
$250 USD in 2 days
2.8
2.8

I understand the importance of following task instructions, evaluating AI-generated solutions objectively, and producing clear, evidence-based results. I am also willing to follow your evaluation guidelines and complete a practical technical assessment to demonstrate my capabilities. My approach to this project is to review each task carefully, understand the expected outcome, and test AI-generated solutions in a terminal environment. I will use Linux command-line tools, Git, relevant programming languages, automated test suites, and debugging techniques to identify errors and verify results. I will distinguish genuine coding issues from environment or configuration problems and document findings in a clear, accurate, and reproducible manner while respecting confidentiality requirements. I can work independently in a remote contract arrangement and focus on delivering reliable, consistent, and high-quality evaluations. I understand the importance of benchmark accuracy, test reliability, troubleshooting failed executions, and maintaining proper technical documentation. I am ready to discuss your workflow, availability, and project requirements and to undertake the technical assessment needed to demonstrate my suitability for this role. Best Regards YAsir k
$340 USD in 3 days
2.7
2.7

Hi there, My hands-on background in AI integration, AI agents, LLM and RAG development, combined with Python, Git, and backend engineering experience, aligns closely with terminal-based AI evaluation work. I can read and assess generated code, work across software environments, and distinguish implementation issues from integration or configuration problems. My technical stack includes Python, JavaScript, TypeScript, Node, Express, Django, FastAPI, REST APIs, MySQL, PostgreSQL, MongoDB, and Git. This gives me a strong foundation for reviewing coding tasks, running and interpreting tests, troubleshooting failed executions, and documenting clear, evidence-based findings. My experience in AI model development and engineering also supports careful analysis of agent behavior and output quality. I would be glad to contribute as a reliable AI evaluation specialist and demonstrate my technical judgment through your assessment process.
$250 USD in 7 days
2.7
2.7

Hi, The real challenge in Terminal-Bench evaluation is separating an agent’s genuine reasoning or coding failure from a broken environment, flaky test, or incorrect task setup. That requires hands-on terminal debugging, not just judging the final output. My approach: 1. Reproduce each task in a clean Linux environment using Git, Python/scripting, shell tools, and Docker where needed. 2. Trace failures through commands, logs, test output, dependencies, and environment state, then distinguish agent errors from infrastructure issues. 3. Record evidence-based, reproducible evaluations with clear reasoning and consistent grading criteria. I have practical experience debugging software in Linux/CLI environments, reviewing AI-generated code, running automated tests, and diagnosing failures across dependencies, permissions, configuration, and runtime behavior. A quick reliability check I use is rerunning failures from a clean state before assigning blame to the model. For similar benchmark work, are your evaluation environments containerized with Docker, and do you provide a fixed test harness for each Terminal-Bench task?
$500 USD in 7 days
2.5
2.5

Hello, "TERMINAL AGENT EVALUATION" - You need someone who can reliably judge AI agent behavior in command-line environments, not just run tests but explain why a solution failed. My background is in Linux, Git, Python, and scripting, with hands-on work debugging software, reading logs, and separating real solution errors from environment or setup issues. I'm comfortable working inside containerized setups and automated test suites, tracing failures back to root cause rather than guessing. For evaluation work I focus on reproducibility: documenting each step, the expected versus actual output, and the exact reason a task passed or failed, so results hold up on review. I treat task instructions and grading criteria as the source of truth and flag ambiguity instead of assuming. I'm available to start promptly and can work independently on a remote, project basis, reporting findings clearly as I go. Best regards Ankit
$300 USD in 3 days
2.1
2.1

Hi , I have a strong background in software development and AI/ML, with hands-on experience in Python, debugging, Git, and building AI-powered applications. I understand the importance of testing solutions carefully, investigating failures, and providing clear, evidence-based technical assessments. My experience includes AI/ML development, API integration, and reviewing AI-assisted code to verify changes rather than blindly accepting generated output. I’m comfortable analyzing code, tracing errors, checking test results, and documenting findings clearly. I want to be transparent: I cannot claim prior Outlier AI or Terminal-Bench project experience. However, my software engineering and AI/ML background gives me a solid foundation for this work, and I’m willing to demonstrate my skills through a practical assessment. I am available for remote contract work and would be happy to discuss your evaluation process, expected hours, and contract terms.
$500 USD in 7 days
2.2
2.2

Hi there, Which Terminal-Bench scenarios are most important here, agentic coding fixes, shell debugging, or full task verification? Also, do you want the review to focus more on reproducible test evidence or on classifying environment issues versus model mistakes? The main challenge is separating a real agent failure from a flaky harness, missing dependency, or bad setup. My approach is to replay the terminal steps, compare outputs against the task spec, and log the exact command, exit code, and failing assertion before judging the result. The risky part is false confidence from partial runs or noisy logs. To avoid that, I would keep a strict checklist for environment state, test commands, and expected artifacts so each evaluation stays reproducible and easy to audit. With terminal workflows, Git, Linux, and scripting in the loop, I can review results carefully and document findings in clear English. Should the funniest failure get more attention than the most important one, or should they both be judged like serious adults? If the scope fits, the first pass can also surface recurring failure patterns for later benchmark cleanup. Best, Hlib
$555 USD in 6 days
2.0
2.0

Hi, I’m interested in contributing to your AI evaluation projects, particularly terminal-based coding tasks that require systematic debugging, reproducible testing, and evidence-based assessment. My proposed approach is to inspect task requirements and evaluation criteria, run solutions in the specified Linux environment, verify results against automated tests, and investigate failures to distinguish genuine code defects from configuration or environment issues. I’d document findings clearly and ensure results can be reproduced. I’m comfortable discussing workflows involving Linux, Git, scripting, and software testing, and I’m willing to complete a practical assessment to demonstrate my technical approach. My application would be strongest with confirmed details of my hands-on experience with Outlier AI and Terminal-Bench, so I’d be happy to provide relevant examples where applicable. Could you share the expected weekly workload, evaluation volume, and whether the contract begins with a paid technical assessment? Best regards, Dzianis Charkasau
$500 USD in 7 days
2.0
2.0

Apex, United States
Payment method verified
Member since Apr 6, 2026
$250-750 USD
$2-8 USD / hour
$250-750 USD
$30-250 USD
$30-250 USD
₹1500-12500 INR
$8-15 AUD / hour
₹12500-37500 INR
€30-250 EUR
₹5000-15000 INR
₹1500-12500 INR
$30-250 USD
$15-25 USD / hour
₹1500-12500 INR
₹12500-37500 INR
₹12500-37500 INR
$10-15 USD / hour
₹1500-12500 INR
$250-750 USD
$150-200 USD
$250-750 USD
$250-750 CAD
₹1500-12500 INR
₹1250-2500 INR / hour
$15-25 USD / hour