grepai-search-boosting

SKILL.md

GrepAI Search Boosting

This skill covers configuring score boosting to prioritize relevant code paths and deprioritize tests, docs, and vendor code.

When to Use This Skill

Prioritizing source code over tests

Penalizing vendor/third-party code

Boosting important directories

Customizing result ranking

What is Boosting?

Boosting modifies search scores based on file paths:

Original score: 0.85 (src/auth.go)
Bonus (+10%):   0.935

Original score: 0.85 (tests/auth_test.go)
Penalty (-50%): 0.425

This ensures production code ranks higher than tests with similar content.

## Configuration

### Basic Configuration

```yaml
# .grepai/config.yaml
search:
  boost:
    enabled: true
    penalties:
      - pattern: /tests/
        factor: 0.5
    bonuses:
      - pattern: /src/
        factor: 1.1

Full Configuration

search:
  boost:
    enabled: true

    # Reduce scores (factor < 1.0)
    penalties:
      # Test files
      - pattern: /tests/
        factor: 0.5
      - pattern: /__tests__/
        factor: 0.5
      - pattern: _test.
        factor: 0.5
      - pattern: .spec.
        factor: 0.5
      - pattern: .test.
        factor: 0.5

      # Documentation
      - pattern: /docs/
        factor: 0.6
      - pattern: /documentation/
        factor: 0.6

      # Vendor/third-party
      - pattern: /vendor/
        factor: 0.3
      - pattern: /node_modules/
        factor: 0.3
      - pattern: /third_party/
        factor: 0.3

      # Generated code
      - pattern: /generated/
        factor: 0.4
      - pattern: .gen.
        factor: 0.4
      - pattern: .pb.go
        factor: 0.4

      # Examples and samples
      - pattern: /examples/
        factor: 0.7
      - pattern: /samples/
        factor: 0.7

    # Increase scores (factor > 1.0)
    bonuses:
      # Core source code
      - pattern: /src/
        factor: 1.1
      - pattern: /lib/
        factor: 1.1
      - pattern: /app/
        factor: 1.1
      - pattern: /core/
        factor: 1.2
      - pattern: /internal/
        factor: 1.1

      # Important directories
      - pattern: /services/
        factor: 1.1
      - pattern: /handlers/
        factor: 1.1
      - pattern: /controllers/
        factor: 1.1

How Factors Work

Factor
Effect
Use Case

0.3
70% reduction
Strong penalty (vendor)

0.5
50% reduction
Moderate penalty (tests)

0.7
30% reduction
Mild penalty (examples)

1.0
No change
Neutral

1.1
10% increase
Mild boost (src)

1.2
20% increase
Moderate boost (core)

1.5
50% increase
Strong boost

Pattern Matching

Patterns match against the full file path:

/project/src/auth/middleware.go
         ^^^^
         Matches "/src/" pattern

Pattern Types

Pattern
Matches
Doesn't Match

/tests/
src/tests/auth.go
tests.go

_test.
auth_test.go
test_auth.go

.spec.
auth.spec.ts
spec/auth.ts

/src/
project/src/main.go
resource/file.go

Effect on Rankings

Without Boosting

Score: 0.85 | tests/auth_test.go:10-30
Score: 0.82 | src/auth/middleware.go:15-45
Score: 0.80 | src/auth/jwt.go:23-55

With Boosting

penalties:
  - pattern: /tests/
    factor: 0.5
bonuses:
  - pattern: /src/
    factor: 1.1

Score: 0.90 | src/auth/middleware.go:15-45  (0.82 × 1.1)
Score: 0.88 | src/auth/jwt.go:23-55        (0.80 × 1.1)
Score: 0.43 | tests/auth_test.go:10-30     (0.85 × 0.5)

Common Configurations

Standard (Recommended)

search:
  boost:
    enabled: true
    penalties:
      - pattern: /tests/
        factor: 0.5
      - pattern: _test.
        factor: 0.5
      - pattern: .spec.
        factor: 0.5
      - pattern: /vendor/
        factor: 0.3
      - pattern: /docs/
        factor: 0.6
    bonuses:
      - pattern: /src/
        factor: 1.1
      - pattern: /lib/
        factor: 1.1

Frontend Project

search:
  boost:
    enabled: true
    penalties:
      - pattern: /__tests__/
        factor: 0.5
      - pattern: .test.
        factor: 0.5
      - pattern: .spec.
        factor: 0.5
      - pattern: /node_modules/
        factor: 0.3
      - pattern: .stories.
        factor: 0.6
      - pattern: /storybook/
        factor: 0.6
    bonuses:
      - pattern: /src/
        factor: 1.1
      - pattern: /components/
        factor: 1.1
      - pattern: /hooks/
        factor: 1.1

Go Project

search:
  boost:
    enabled: true
    penalties:
      - pattern: _test.go
        factor: 0.5
      - pattern: _mock.go
        factor: 0.5
      - pattern: /testdata/
        factor: 0.5
      - pattern: /vendor/
        factor: 0.3
      - pattern: .pb.go
        factor: 0.4
    bonuses:
      - pattern: /internal/
        factor: 1.1
      - pattern: /cmd/
        factor: 1.1
      - pattern: /pkg/
        factor: 1.1

Python Project

search:
  boost:
    enabled: true
    penalties:
      - pattern: /tests/
        factor: 0.5
      - pattern: test_
        factor: 0.5
      - pattern: _test.py
        factor: 0.5
      - pattern: /conftest
        factor: 0.5
      - pattern: /fixtures/
        factor: 0.6
    bonuses:
      - pattern: /src/
        factor: 1.1
      - pattern: /app/
        factor: 1.1
      - pattern: /core/
        factor: 1.2

Monorepo

search:
  boost:
    enabled: true
    penalties:
      - pattern: /tests/
        factor: 0.5
      - pattern: _test.
        factor: 0.5
      - pattern: /packages/deprecated/
        factor: 0.3
      - pattern: /packages/legacy/
        factor: 0.4
    bonuses:
      - pattern: /packages/core/
        factor: 1.2
      - pattern: /packages/api/
        factor: 1.1
      - pattern: /packages/shared/
        factor: 1.1

Disabling Boosting

To disable boosting entirely:

search:
  boost:
    enabled: false

Or remove the boost section from config.

Boosting vs Ignoring

Approach
Effect
Use Case

Ignore
Completely excluded
Dependencies, build output

Penalty
Still searchable, lower rank
Tests, docs, examples

Neutral
Default ranking
Regular source code

Bonus
Higher rank
Core business logic

When to Ignore vs Penalize

Ignore: Files you NEVER want to search (node_modules, .git)

Penalize: Files you RARELY want but might need (tests, docs)

Testing Your Configuration

After configuring boosting:

# Search and observe rankings
grepai search "authentication"

# Check if tests are properly deprioritized
grepai search "test authentication"  # Should still find tests, but ranked lower

Best Practices

Start with penalties: Deprioritize tests/vendor first

Add bonuses sparingly: Only for truly important paths

Test with real queries: Verify results make sense

Don't over-penalize: 0.5 is usually enough for tests

Document your choices: Add comments in config

Common Issues

❌ Problem: Tests always show up first
✅ Solution: Add penalty patterns for your test naming convention

❌ Problem: Can't find code in penalized paths
✅ Solution: Penalties reduce rank, don't hide. Use ignore for complete exclusion.

❌ Problem: Scores above 1.0 seem wrong
✅ Solution: Bonuses can push scores above 1.0; this is normal

❌ Problem: Pattern not matching
✅ Solution: Check that pattern appears in full path (use /tests/ not just tests)

Output Format

Boosting configuration status:

✅ Search Boosting Configured

   Status: Enabled

   Penalties (5):
   - /tests/      → 0.5 (50% reduction)
   - _test.       → 0.5
   - .spec.       → 0.5
   - /vendor/     → 0.3 (70% reduction)
   - /docs/       → 0.6

   Bonuses (3):
   - /src/        → 1.1 (10% boost)
   - /lib/        → 1.1
   - /core/       → 1.2 (20% boost)

   Effect: Source code ranks higher than tests with similar content