scholar-evaluation

SKILL.md

Scholar Evaluation

Use this skill to evaluate academic or scientific work with a repeatable rubric.

When to Use

Reviewing a research paper, proposal, thesis chapter, or literature review.

Checking whether claims are supported by cited evidence.

Evaluating methodology, study design, analysis, or limitations.

Comparing two or more papers for quality or relevance.

Producing structured feedback for revision.

Evaluation Scope

Start by identifying the artifact:

empirical research paper

theoretical paper

technical report

systematic or narrative literature review

research proposal

thesis or dissertation chapter

conference abstract or short paper

Then choose scope:

comprehensive: all rubric dimensions

targeted: one or two dimensions, such as method or citations

comparative: rank multiple works against the same rubric

Rubric

Score each applicable dimension from 1 to 5:

5: excellent; clear, rigorous, and publication-ready

4: good; minor improvements needed

3: adequate; meaningful gaps but usable

2: weak; substantial revision needed

1: poor; major validity or clarity problems

Use N/A for dimensions that do not apply.

1. Problem and Research Question

Is the problem clear and specific?

Is the contribution meaningful?

Are scope and assumptions explicit?

Does the question match the claimed contribution?

2. Literature and Context

Is relevant prior work covered?

Does the work synthesize rather than merely list sources?

Are gaps accurately identified?

Are recent and foundational sources balanced?

3. Methodology

Does the method answer the research question?

Are design choices justified?

Are variables, datasets, participants, or materials described clearly?

Could another researcher reproduce the work?

Are ethical and practical constraints acknowledged?

4. Data and Evidence

Are data sources credible and appropriate?

Is sample size or corpus coverage adequate?

Are inclusion, exclusion, and preprocessing decisions documented?

Are missing data and bias risks discussed?

5. Analysis

Are statistical, qualitative, or computational methods appropriate?

Are baselines and controls fair?

Are uncertainty, sensitivity, or robustness checks included when needed?

Are alternative explanations considered?

6. Results and Interpretation

Are results clearly presented?

Do claims stay within the evidence?

Are figures, tables, and metrics understandable?

Are negative or null results handled honestly?

7. Limitations and Threats to Validity

Are limitations specific rather than generic?

Are internal, external, construct, and conclusion-validity risks addressed?

Does the paper distinguish speculation from demonstrated results?

8. Writing and Structure

Is the argument easy to follow?

Are sections organized around the research question?

Are definitions and notation clear?

Is the tone precise and scholarly?

9. Citations

Do cited papers support the claims attached to them?

Are primary sources used where possible?

Are reviews labeled as reviews?

Are preprints labeled as preprints?

Are citation metadata and links correct?

Review Process

Read the abstract, introduction, figures, and conclusion for claimed
contribution.

Read methods and results for evidence quality.

Check the strongest claims against cited sources.

Score each applicable dimension.

Separate critical blockers from revision suggestions.

End with concrete next edits.

Output Template

# Scholar Evaluation: <Artifact>

## Overall Assessment

- Overall score: <1-5 or N/A>
- Confidence: <high | medium | low>
- Summary: <3-5 sentences>

## Dimension Scores

| Dimension | Score | Evidence | Revision priority |
| --- | ---: | --- | --- |
| Problem and question |  |  |  |
| Literature and context |  |  |  |
| Methodology |  |  |  |
| Data and evidence |  |  |  |
| Analysis |  |  |  |
| Results and interpretation |  |  |  |
| Limitations |  |  |  |
| Writing and structure |  |  |  |
| Citations |  |  |  |

## Critical Issues

## Recommended Revisions

## Evidence Checks Needed

Pitfalls

Do not use the score as a substitute for concrete feedback.

Do not penalize a paper for omitting a dimension outside its scope.

Do not treat citation count, venue, or author reputation as proof of quality.

Do not accept unsupported claims just because they appear in the abstract.