Skip to main content

Regulation Data Sources - Research Summary

Available Data Sources​

✅ APIs Available​

  1. Fish Rules API
    • URL: https://fish.management
    • Type: Commercial API
    • Coverage: Multiple states (saltwater focus)
    • Access: May require API key/partnership
    • Update Frequency: Real-time
    • Format: JSON API
    • Best For: Saltwater regulations, multiple states

⚠️ Web Scraping Required​

Most state DNR websites do NOT have public APIs. Regulations are published as:

  • PDF documents (fishing guides)
  • HTML web pages
  • Interactive maps

States Requiring Web Scraping:

Legal Considerations:

  • Check robots.txt
  • Review Terms of Service
  • Respect rate limits
  • Include attribution if required

❌ No Public APIs​

  • NOAA Fisheries: No public API for regulations (provides resources/links)
  • Federal Database: No centralized federal database for state regulations
  • USFWS: No public API for fishing regulations

Phase 1: API Integration (Immediate)​

  1. Fish Rules API: Integrate for states they cover
  2. Contact State DNRs: Request API access or partnerships
  3. Manual Entry: For critical states without APIs

Phase 2: Web Scraping (Short-term)​

  1. Start with PDF Parsing: Many states publish PDF guides
  2. HTML Scraping: For states with structured HTML
  3. Monitor for Changes: Set up alerts for website changes

Phase 3: Partnerships (Long-term)​

  1. State Partnerships: Work with state DNRs for official data
  2. Data Sharing Agreements: Formal agreements for data access
  3. API Development: Help states develop APIs

Data Source Configuration Examples​

Fish Rules API​

{
    "SourceName": "Fish Rules API",
    "SourceType": "Api",
    "ApiUrl": "https://api.fish.management/v1/regulations/{state}",
    "UpdateFrequency": "Weekly",
    "RateLimitRequestsPerMinute": 60
}

Michigan DNR Web Scraping​

{
    "SourceName": "Michigan DNR Website",
    "SourceType": "WebScraping",
    "ScrapingUrl": "https://www.michigan.gov/dnr/managing-resources/laws/fishing",
    "UpdateFrequency": "Monthly",
    "RequiresAttribution": true,
    "AttributionText": "Regulations from Michigan DNR"
}

PDF Import​

{
    "SourceName": "State Fishing Guide PDF",
    "SourceType": "FileImport",
    "FileUrl": "https://dnr.state.gov/fishing-guide-2024.pdf",
    "FileFormat": "PDF",
    "UpdateFrequency": "Annually"
}

Implementation Notes​

  • Rate Limiting: Implement rate limiting for all sources
  • Error Handling: Robust error handling for network issues
  • Data Validation: Validate parsed data before saving
  • Change Detection: Detect when regulations change
  • User Notifications: Notify users of regulation changes
  • Compliance: Ensure ToS compliance for all sources