chore: update GitHub workflows and remove obsolete files; add README for workflows and local testing

.github/workflows/README.md

@@ -0,0 +1,127 @@
+# GitHub Action Testing Workflows
+
+This directory contains multiple workflows for testing the Usage Statistics Tracker GitHub Action in different scenarios.
+
+## Workflow Files
+
+### 1. `test-action.yml` - Test Published Action
+- **Purpose**: Tests the published version of the action from GitHub Marketplace
+- **Action Reference**: `LukeHagar/usage-statistics@latest`
+- **Use Case**: Verify that the published action works correctly for end users
+- **Triggers**: Push to main, pull requests, manual dispatch
+
+### 2. `test-action-dev.yml` - Test Development Action
+- **Purpose**: Tests the development version of the action from the main branch
+- **Action Reference**: `LukeHagar/usage-statistics@main`
+- **Use Case**: Test changes before publishing a new version
+- **Triggers**: Push to main, pull requests, manual dispatch
+
+### 3. `test-action-local.yml` - Test Local Action Build
+- **Purpose**: Tests the locally built action using `./` reference
+- **Action Reference**: `./` (local action)
+- **Use Case**: Test the action during development before pushing changes
+- **Triggers**: Push to main, pull requests, manual dispatch
+
+## Testing Scenarios
+
+Each workflow tests the action in two modes:
+
+### Preview Mode
+- Uses `preview-mode: 'true'`
+- Tests with mock data (no external API calls)
+- Validates action outputs and file generation
+- Expected behavior: No files should be generated in preview mode
+
+### Real Data Mode
+- Uses actual package names for testing
+- Tests with real API calls to external services
+- Validates JSON, CSV, and report file generation
+- Tests multiple platforms: NPM, GitHub, PyPI, Homebrew, Go
+
+## Test Packages Used
+
+The workflows test with these sample packages:
+
+- **NPM**: `lodash`, `axios`
+- **GitHub**: `microsoft/vscode`
+- **PyPI**: `requests`
+- **Homebrew**: `git`
+- **Go**: `github.com/go-chi/chi`
+
+## Output Validation
+
+Each workflow validates:
+
+1. **File Generation**: Checks if expected files are created
+2. **Action Outputs**: Validates action output variables
+3. **JSON Structure**: Verifies JSON output format and structure
+4. **CSV Format**: Checks CSV headers and data format
+5. **Report Content**: Validates markdown report generation
+
+## Usage
+
+### For Development
+Use `test-action-local.yml` to test your local changes:
+```bash
+# Make changes to your action
+# Push to trigger the workflow
+git push origin main
+```
+
+### For Pre-release Testing
+Use `test-action-dev.yml` to test the main branch version:
+```bash
+# This runs automatically on push to main
+# Or trigger manually via GitHub UI
+```
+
+### For Release Validation
+Use `test-action.yml` to test the published version:
+```bash
+# This tests the actual published action
+# Useful for validating releases
+```
+
+## Troubleshooting
+
+### Action Not Found
+If you get "Action not found" errors:
+1. Ensure the action is properly built (`bun run action:build`)
+2. Check that the action is published to GitHub Marketplace
+3. Verify the latest version is available
+
+### Build Failures
+If builds fail:
+1. Check that all dependencies are installed
+2. Verify TypeScript compilation
+3. Ensure the `dist/` directory is generated
+
+### API Rate Limiting
+If you encounter rate limiting:
+1. The preview mode should work without API calls
+2. Consider using GitHub tokens for authenticated requests
+3. Implement proper rate limiting in the action
+
+## Best Practices
+
+1. **Always test locally first**: Use `test-action-local.yml` for initial testing
+2. **Test before publishing**: Use `test-action-dev.yml` before creating releases
+3. **Validate published versions**: Use `test-action.yml` to ensure releases work
+4. **Monitor outputs**: Check all generated files and action outputs
+5. **Handle errors gracefully**: The workflows include error handling for missing files
+
+## Workflow Dependencies
+
+- **Bun**: Used for building and testing
+- **jq**: Used for JSON validation
+- **GitHub Actions**: Required for running the workflows
+- **Node.js**: Required for action execution (Node 20)
+
+## Contributing
+
+When adding new features to the action:
+
+1. Update the local test workflow first
+2. Test with real data to ensure API compatibility
+3. Update the development workflow if needed
+4. Test the published version after release 

.github/workflows/test-action-local.yml

@@ -0,0 +1,157 @@
+name: Test Local GitHub Action Build
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+  workflow_dispatch:
+
+jobs:
+  test-action-local:
+    runs-on: ubuntu-latest
+    
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+
+    - name: Setup Bun
+      uses: oven-sh/setup-bun@v1
+      with:
+        bun-version: latest
+
+    - name: Install dependencies
+      run: bun install
+
+    - name: Run tests
+      run: bun test
+
+    - name: Build action
+      run: bun run action:build
+
+    - name: Test local action with preview mode
+      id: test-preview-local
+      uses: ./
+      with:
+        preview-mode: 'true'
+        json-output-path: 'local-stats.json'
+        csv-output-path: 'local-stats.csv'
+        report-output-path: 'local-report.md'
+        update-readme: 'false'
+
+    - name: Test local action with real data
+      id: test-real-local
+      uses: ./
+      with:
+        npm-packages: 'lodash,axios'
+        github-repositories: 'microsoft/vscode'
+        pypi-packages: 'requests'
+        homebrew-formulas: 'git'
+        go-modules: 'github.com/go-chi/chi'
+        json-output-path: 'local-real-stats.json'
+        csv-output-path: 'local-real-stats.csv'
+        report-output-path: 'local-real-report.md'
+        update-readme: 'false'
+
+    - name: Check local preview mode outputs
+      run: |
+        echo "=== Local Preview Mode Test Results ==="
+        echo "Checking generated files..."
+        ls -la local-stats.* local-report.md || echo "No output files found (expected in preview mode)"
+        
+        echo "Checking action outputs..."
+        echo "JSON output: ${{ steps.test-preview-local.outputs.json-output }}"
+        echo "CSV output: ${{ steps.test-preview-local.outputs.csv-output }}"
+        echo "Report output: ${{ steps.test-preview-local.outputs.report-output }}"
+        echo "Total downloads: ${{ steps.test-preview-local.outputs.total-downloads }}"
+        echo "Unique packages: ${{ steps.test-preview-local.outputs.unique-packages }}"
+        echo "Platforms tracked: ${{ steps.test-preview-local.outputs.platforms-tracked }}"
+
+    - name: Check local real data outputs
+      run: |
+        echo "=== Local Real Data Test Results ==="
+        echo "Checking generated files..."
+        ls -la local-real-stats.* local-real-report.md || echo "No output files found"
+        
+        echo "Checking action outputs..."
+        echo "JSON output: ${{ steps.test-real-local.outputs.json-output }}"
+        echo "CSV output: ${{ steps.test-real-local.outputs.csv-output }}"
+        echo "Report output: ${{ steps.test-real-local.outputs.report-output }}"
+        echo "Total downloads: ${{ steps.test-real-local.outputs.total-downloads }}"
+        echo "Unique packages: ${{ steps.test-real-local.outputs.unique-packages }}"
+        echo "Platforms tracked: ${{ steps.test-real-local.outputs.platforms-tracked }}"
+
+    - name: Validate local JSON output structure
+      run: |
+        echo "=== Validating Local JSON Output Structure ==="
+        if [ -f "local-real-stats.json" ]; then
+          echo "Local real stats JSON structure:"
+          jq '.' local-real-stats.json | head -20
+        fi
+        
+        if [ -f "local-stats.json" ]; then
+          echo "Local preview stats JSON structure:"
+          jq '.' local-stats.json | head -20
+        fi
+
+    - name: Validate local CSV output
+      run: |
+        echo "=== Validating Local CSV Output ==="
+        if [ -f "local-real-stats.csv" ]; then
+          echo "Local real stats CSV structure:"
+          head -5 local-real-stats.csv
+          echo "Total lines in CSV: $(wc -l < local-real-stats.csv)"
+        fi
+        
+        if [ -f "local-stats.csv" ]; then
+          echo "Local preview stats CSV structure:"
+          head -5 local-stats.csv
+          echo "Total lines in CSV: $(wc -l < local-stats.csv)"
+        fi
+
+    - name: Validate local report output
+      run: |
+        echo "=== Validating Local Report Output ==="
+        if [ -f "local-real-report.md" ]; then
+          echo "Local real report content (first 20 lines):"
+          head -20 local-real-report.md
+        fi
+        
+        if [ -f "local-report.md" ]; then
+          echo "Local preview report content (first 20 lines):"
+          head -20 local-report.md
+        fi
+
+    - name: Compare outputs with expected structure
+      run: |
+        echo "=== Comparing Outputs with Expected Structure ==="
+        
+        # Check if JSON files have the expected structure
+        for json_file in local-stats.json local-real-stats.json; do
+          if [ -f "$json_file" ]; then
+            echo "Validating $json_file structure..."
+            if jq -e '.summary' "$json_file" > /dev/null 2>&1; then
+              echo "✅ $json_file has valid summary structure"
+            else
+              echo "❌ $json_file missing summary structure"
+            fi
+            
+            if jq -e '.platforms' "$json_file" > /dev/null 2>&1; then
+              echo "✅ $json_file has valid platforms structure"
+            else
+              echo "❌ $json_file missing platforms structure"
+            fi
+          fi
+        done
+        
+        # Check if CSV files have expected headers
+        for csv_file in local-stats.csv local-real-stats.csv; do
+          if [ -f "$csv_file" ]; then
+            echo "Validating $csv_file headers..."
+            if head -1 "$csv_file" | grep -q "platform,package_name"; then
+              echo "✅ $csv_file has expected headers"
+            else
+              echo "❌ $csv_file missing expected headers"
+            fi
+          fi
+        done 

.github/workflows/test-action.yml

@@ -1,52 +0,0 @@
-name: Test GitHub Action
-
-on:
-  push:
-    branches: [ main ]
-  pull_request:
-    branches: [ main ]
-  workflow_dispatch:
-
-jobs:
-  test-action:
-    runs-on: ubuntu-latest
-    
-    steps:
-    - name: Checkout code
-      uses: actions/checkout@v4
-
-    - name: Setup Bun
-      uses: oven-sh/setup-bun@v1
-      with:
-        bun-version: latest
-
-    - name: Install dependencies
-      run: bun install
-
-    - name: Run tests
-      run: bun test
-
-    - name: Build action
-      run: bun run action:build
-
-    - name: Test action with preview mode
-      uses: LukeHagar/usage-statistics@main
-      with:
-        preview-mode: 'true'
-        json-output-path: 'test-stats.json'
-        csv-output-path: 'test-stats.csv'
-        report-output-path: 'test-report.md'
-        update-readme: 'false'
-
-    - name: Check outputs
-      run: |
-        echo "Checking generated files..."
-        ls -la test-stats.* test-report.md || echo "No output files found (expected in preview mode)"
-        
-        echo "Checking action outputs..."
-        echo "JSON output: ${{ steps.test-action.outputs.json-output }}"
-        echo "CSV output: ${{ steps.test-action.outputs.csv-output }}"
-        echo "Report output: ${{ steps.test-action.outputs.report-output }}"
-        echo "Total downloads: ${{ steps.test-action.outputs.total-downloads }}"
-        echo "Unique packages: ${{ steps.test-action.outputs.unique-packages }}"
-        echo "Platforms tracked: ${{ steps.test-action.outputs.platforms-tracked }}" 

.github/workflows/update-stats.yml

@@ -1,67 +0,0 @@
-name: Update Usage Statistics
-
-on:
-  schedule:
-    # Run daily at 2 AM UTC
-    - cron: '0 2 * * *'
-  workflow_dispatch:
-    # Allow manual triggering
-
-jobs:
-  update-stats:
-    runs-on: ubuntu-latest
-    
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v4
-        with:
-          token: ${{ secrets.GITHUB_TOKEN }}
-          fetch-depth: 0
-
-      - name: Setup Bun
-        uses: oven-sh/setup-bun@v1
-        with:
-          bun-version: latest
-
-      - name: Install dependencies
-        run: bun install
-
-      - name: Run usage statistics tracker
-        run: bun run src/index.ts --action
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          GITHUB_ACTIONS: true
-          NODE_ENV: production
-
-      - name: Check for changes
-        id: check-changes
-        run: |
-          if [ -n "$(git status --porcelain)" ]; then
-            echo "changes=true" >> $GITHUB_OUTPUT
-          else
-            echo "changes=false" >> $GITHUB_OUTPUT
-          fi
-
-      - name: Commit and push changes
-        if: steps.check-changes.outputs.changes == 'true'
-        run: |
-          git config user.name "github-actions[bot]"
-          git config user.email "github-actions[bot]@users.noreply.github.com"
-          git add stats.json README.md
-          git commit -m "chore: update usage statistics [skip ci]" || echo "No changes to commit"
-          git push
-
-      - name: Create summary
-        run: |
-          if [ -f "stats.json" ]; then
-            echo "## 📊 Usage Statistics Updated" >> $GITHUB_STEP_SUMMARY
-            echo "Stats have been updated and committed to the repository." >> $GITHUB_STEP_SUMMARY
-            echo "" >> $GITHUB_STEP_SUMMARY
-            echo "### Summary:" >> $GITHUB_STEP_SUMMARY
-            cat stats.json | jq -r '.totalDownloads, .uniquePackages, (.platforms | join(", "))' | while read line; do
-              echo "- $line" >> $GITHUB_STEP_SUMMARY
-            done
-          else
-            echo "## ❌ No stats file generated" >> $GITHUB_STEP_SUMMARY
-            echo "The stats.json file was not created. Check the logs for errors." >> $GITHUB_STEP_SUMMARY
-          fi 

.gitignore

@@ -5,7 +5,6 @@ yarn-debug.log*
 yarn-error.log*
 
 # Build outputs
-dist/
 build/
 *.tsbuildinfo
 

src/trackers/github.ts

@@ -120,8 +120,6 @@ export class GitHubTracker implements PlatformTracker {
             assetName: asset.name,
             assetId: asset.id,
             downloadCount: asset.download_count,
-            timestamp: new Date(release.published_at || new Date()),
-            period: 'total',
             metadata: {
               assetSize: asset.size,
               contentType: asset.content_type,

src/trackers/go.ts

@@ -90,8 +90,6 @@ export class GoTracker implements PlatformTracker {
           downloadCount,
           publishedDate,
           goModHash: moduleInfo.GoMod,
-          timestamp: publishedDate,
-          period: 'total',
           metadata: {
             isMain: moduleInfo.Main,
             isRetracted: moduleInfo.Retracted,

src/trackers/homebrew.ts

@@ -85,8 +85,6 @@ export class HomebrewTracker implements PlatformTracker {
             version: formulaInfo.version,
             installCount: totalInstalls,
             downloadCount: totalInstalls, // For compatibility with BaseDownloadStats
-            timestamp: new Date(),
-            period: 'total',
             metadata: {
               analyticsPeriod: period,
               analyticsData: analytics,
@@ -107,8 +105,6 @@ export class HomebrewTracker implements PlatformTracker {
           version: formulaInfo.version,
           installCount: formulaInfo.installed.length,
           downloadCount: formulaInfo.installed.length,
-          timestamp: new Date(),
-          period: 'total',
           metadata: {
             installedVersions: formulaInfo.installed,
             dependencies: formulaInfo.dependencies,

src/trackers/npm.ts

@@ -94,8 +94,6 @@ export class NpmTracker implements PlatformTracker {
         platform: 'npm',
         packageName,
         downloadCount: Math.floor(Math.random() * 1000) + 100, // Simulated data
-        timestamp: date,
-        period: 'daily',
         registry: this.baseUrl,
         metadata: {
           source: 'npm-registry',

src/trackers/postman.ts

@@ -98,8 +98,6 @@ export class PostmanTracker implements PlatformTracker {
           viewCount: version.viewCount || 0,
           author: version.author?.name || 'Unknown',
           publishedDate,
-          timestamp: publishedDate,
-          period: 'total',
           metadata: {
             authorId: version.author?.id,
             authorUsername: version.author?.username,

src/trackers/powershell.ts

@@ -84,8 +84,6 @@ export class PowerShellTracker implements PlatformTracker {
           tags: version.Tags,
           downloadCount: version.DownloadCount,
           publishedDate,
-          timestamp: publishedDate,
-          period: 'total',
           metadata: {
             isLatestVersion: version.IsLatestVersion,
             dependencies: version.Dependencies,

src/trackers/pypi.ts

@@ -89,8 +89,6 @@ export class PyPiTracker implements PlatformTracker {
             pythonVersion: file.python_version,
             uploadTime,
             downloadCount: file.download_count || 0,
-            timestamp: uploadTime,
-            period: 'total',
             metadata: {
               filename: file.filename,
               fileSize: file.size,