Copy of Software Developer – AI Coding Assistant Evaluation

We are looking for developers who have a software project they personally own, such as a side project, personal tool, or open-source library.

Responsibilities

• In this project, you will use an AI coding assistant to work on your existing software project.
• This may involve building new features, fixing bugs, refactoring code, or writing documentation.
• As you work, you will be presented with two versions of the AI assistant's response and asked to select the one that works better, along with a short explanation of your choice.
• Bring and work on a software project that you personally own and maintain in a Git repository.
• Use an AI coding assistant to build features, fix bugs, refactor code, and improve documentation.
• Review and compare two versions of AI-generated coding responses.
• Evaluate which response is more useful, accurate, and effective for your project.
• Provide clear, honest feedback explaining why you selected one response over the other.
• Work comfortably with terminal commands and Git during project sessions.
• Follow project guidelines and ensure that no passwords, API keys, or other people's personal information are included in your sessions.

Requirements

• Experienced software developer with a personal software project that you own and maintain in a Git repository.
• Comfortable working with terminal commands and Git.
• Has a computer running macOS, Linux, or Windows with WSL2.
• Able to review and evaluate AI-generated code and coding suggestions.
• Strong written communication skills, with the ability to explain decisions clearly and briefly.
• Able to provide independent and honest feedback without using AI assistance to write evaluation responses.
• Open-source projects must have an approved license.

Preferred

• Experience maintaining open-source software projects.
• Experience building personal tools, applications, libraries, or other software projects.
• Familiarity with AI coding assistants and developer productivity tools.
• Experience with software debugging, feature development, code refactoring, and documentation.
• Worldwide applicants are welcome, subject to project eligibility.

Pay

• Up to $100 per hour.
• Work volume may fluctuate throughout the contract.

Benefits

• Fully remote freelance opportunity.
• Payments will be released weekly as a lump sum for work completed during the previous week.

Expected Workload

• Approximately 10–39 hours per week, depending on project needs.
• AI usage during the project will be covered, so candidates will not need to pay for an AI subscription or API usage.

Project Eligibility and Ownership Requirements

Candidates must bring a software project they personally and solely own.

Projects with multiple maintainers may require additional confirmation. Projects owned by an employer, client, university, or startup are not eligible. Projects covered by an NDA are not eligible. Projects involving AI model training or evaluation, machine learning infrastructure, security exploit tooling, or the processing of other people's personal data are not eligible. The client's team must review and approve the project before work begins. Selected candidates must sign contributor terms before starting.

Important Privacy and Participation Notes

Transcripts from project sessions will be used to improve AI models. Once submitted, these transcripts cannot be withdrawn. Only project session transcripts will be shared, not the rest of the candidate's software project. Ratings and written feedback provided by candidates will become part of the project's deliverables. Candidates must not include passwords, API keys, or other people's personal information in their sessions. All ratings and written evaluations must be completed independently, without AI assistance.

Important

An active Upwork contract is required before candidates can receive onboarding instructions, access to the evaluation platform, or payment for the project. Candidates should accept the contract offer once received. Due to the high volume of applicants being contacted, the team may not be able to respond to every individual message. Thank you for your understanding.

An enterprise client is seeking experienced software developers to participate in an AI training project focused on testing and evaluating AI coding assistants. The project aims to improve AI coding tools by gathering feedback from developers as they work on their own software projects and compare different AI-generated solutions.