AI Fluency의 역설: 숙련 사용자가 더 많이 실패하는 이유

# 사용자 fluency 수준을 판단하는 LLM 어노테이션 프롬프트 예시 (논문의 annotation protocol 기반)

system_prompt = """
You are an AI fluency annotator. Analyze the following conversation transcript and evaluate the user's AI fluency level.

Evaluate the following dimensions:

1. Interaction Style:
   - augmentative: User iterates collaboratively, refines goals, critically assesses outputs
   - delegative: User passively accepts AI plans and responses
   - other: Neither pattern dominates

2. Fluency Behaviors (mark all that apply):
   - iterative_refinement: User refines requests based on AI output
   - critical_output_evaluation: User questions or challenges AI responses
   - context_provision: User provides rich background context
   - goal_clarification: User clarifies or sharpens their goals mid-conversation
   - decomposition: User breaks complex tasks into subtasks
   - fact_checking: User verifies AI claims

3. Anti-Fluency Behaviors (mark all that apply):
   - passive_acceptance: User accepts AI output without scrutiny
   - vague_delegation: User gives underspecified instructions
   - over_trust: User shows excessive trust in AI responses
   - prompt_flailing: User makes random changes without clear strategy

4. Overall Fluency Assessment: high | moderate | low | minimal

Respond in JSON format.
"""

user_message = f"""
Transcript:
{conversation_transcript}

Provide your fluency annotation:
"""

# Example output structure:
# {
#   "transcript_summary": "User asked for help debugging a React component",
#   "interaction_style": "augmentative",
#   "fluency_behaviors": [
#     {"behavior": "iterative_refinement", "strength": 3, "evidence": "User provided specific error message and asked follow-up"},
#     {"behavior": "critical_output_evaluation", "strength": 2, "evidence": "User questioned whether suggested fix would cause side effects"}
#   ],
#   "anti_fluency_behaviors": [],
#   "fluency_assessment": "high",
#   "assessment_rationale": "User demonstrated clear augmentative behavior..."
# }