Everyday AI Agent Tester

Employment Type: Contractor
Work Arrangement: Remote
Eligible Location: United States
Education: Bachelor's degree required
Working Environment: Smartphone-based

Compensation

Hourly Rate: $25–$45 per hour

Estimated Time per Task: Approximately 1.5 hours per run, including evidence collection and scoring.

Actual earnings depend on project availability, tasks completed, and hours worked.

About the Role

We are seeking detail-oriented Everyday AI Agent Testers to evaluate how effectively AI agents handle common real-world tasks.

You will test two AI agents side by side using your personal accounts and smartphone. Scenarios may include managing bills, booking trips, shopping, handling returns, following up on emails, organizing family activities, and completing forms.

You will monitor agent actions, approve or stop key steps, record each testing attempt, and score the results. Your feedback will help assess the agents' reliability, task completion, quality, and overall user experience.

No coding or previous AI development experience is required. Familiarity with everyday AI tools and the ability to follow instructions carefully are essential.

Key Responsibilities

Everyday Task Testing

• Test two AI agents across realistic everyday scenarios.

• Run tasks involving bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms.

• Observe how agents interpret instructions and execute multi-step tasks.

• Approve or stop agent actions at designated decision points.

• Assess whether each agent completes the requested task and produces satisfactory results.

Screen Recording & Evidence Collection

• Screen record every testing attempt using a smartphone.

• Capture agent actions and relevant steps throughout each task.

• Ensure recordings provide sufficient evidence for evaluating performance.

• Follow task-specific instructions and document relevant observations.

Performance Evaluation & Feedback

• Score each run based on task completion, quality, control issues, user satisfaction, and time taken.

• Evaluate the overall user experience when interacting with each agent.

• Identify errors, incomplete tasks, and issues requiring user intervention.

• Provide clear written feedback explaining what worked, what did not, and what could improve.

• Maintain accurate, consistent assessments across assigned scenarios.

Required Qualifications

• Must be based in the United States.

• Bachelor's degree.

• Two to three years of professional experience, preferably in engineering, business management, office administration, or operations.

• Existing familiarity with using AI tools in everyday life.

• An active personal account with a supported AI service on the Free, Go, or Plus plan; Pro accounts are not eligible.

• A personal smartphone capable of screen recording.

• Strong attention to detail and ability to follow instructions precisely.

• Clear written communication skills.

• Ability to document task execution and evaluate outcomes objectively.

Preferred Qualifications

• Experience using ChatGPT, Muse, or other AI agents.

• Background in engineering, business management, office administration, or operations.

• Experience with software testing, task evaluation, or documenting user workflows.

• Ability to identify usability issues and explain problems clearly.

Generalists are welcome, while relevant domain expertise is an advantage.

Key Skills

• ChatGPT

• Muse

• AI Agents

• AI Agent Testing

• Task Completion Evaluation

• Instruction Following

• Screen Recording

• Evidence Collection

• Performance Scoring

• User Experience Assessment

• Attention to Detail

• Written Communication

• Process Documentation

• Critical Thinking

• Quality Evaluation

Work Environment

This is a remote, hourly contract opportunity involving smartphone-based testing. Work is performed on your own schedule, subject to project requirements and task availability.

Each testing run takes approximately 1.5 hours, including evidence collection and scoring. You will use your personal accounts and smartphone to test AI agents against realistic everyday scenarios.

Selection Process

1. Submit an application.

2. Complete the screening questions.

3. Attend a 15–30-minute interview.

4. Complete a profile review for potential client presentation.

5. Complete onboarding with the project operations team.

What Success Looks Like

Success means producing clear, consistent, evidence-based evaluations of how AI agents perform everyday tasks. Your assessments should accurately capture completion, quality, control issues, user satisfaction, and time taken, helping identify opportunities to improve agent reliability and usability.