{"id":736,"date":"2025-10-23T15:11:15","date_gmt":"2025-10-23T19:11:15","guid":{"rendered":"https:\/\/scootercam.net\/notes\/?p=736"},"modified":"2025-10-23T15:11:16","modified_gmt":"2025-10-23T19:11:16","slug":"how-scootercam-gets-its-data","status":"publish","type":"post","link":"https:\/\/scootercam.net\/blog\/how-scootercam-gets-its-data\/","title":{"rendered":"How Scootercam gets its data"},"content":{"rendered":"\n

Most of Scootercam’s data comes from data broker Visual Crossing. Visual Crossing collects its weather data from a blend of national meteorological services, global forecast models, radar and satellite sources<\/strong>, and a network of local weather stations<\/strong>. The platform then merges and normalizes these feeds to provide both historical and forecast data<\/strong> at fine spatial and temporal resolution. When Visual Crossing receives Scootercam’s query, it replies with a seemingly-endless JSON file. This is structured and contains thousands of little datapoints, like the expected high temp three days from now. Other scripts will find that number when they need it.<\/p>\n\n\n\n

The JSON output is typically structured with a locations<\/code><\/strong> object containing one or more values<\/code><\/strong> arrays\u2014each item representing an hourly or daily record with fields like datetime<\/code>, temp<\/code>, humidity<\/code>, windspeed<\/code>, precip<\/code>, and weather condition codes or descriptions. A small portion is shown here. <\/p>\n\n\n\n

{\n  \"locations\": {\n    \"Fennville,MI,US\": {\n      \"stationContributions\": [],\n      \"values\": [\n        {\n          \"datetime\": \"2025-10-22\",\n          \"datetimeEpoch\": 1761110400,\n          \"tempmax\": 17.2,\n          \"tempmin\": 9.3,\n          \"temp\": 13.1,\n          \"feelslikemax\": 17.2,\n          \"feelslikemin\": 8.7,\n          \"humidity\": 72.1,\n          \"precip\": 1.2,\n          \"preciptype\": [\"rain\"],\n          \"windspeed\": 14.4,\n          \"winddir\": 238,\n          \"cloudcover\": 68,\n          \"conditions\": \"Rain, Partially cloudy\",\n          \"description\": \"Partly cloudy throughout the day with periods of rain.\",\n          \"icon\": \"rain\",\n          \"sunrise\": \"07:59:00\",\n          \"sunset\": \"18:54:00\"\n        }\n      ],\n      \"id\": \"Fennville,MI,US\",\n      \"latitude\": 42.5931,\n      \"longitude\": -86.1048,\n      \"address\": \"Fennville, MI, United States\",\n      \"timezone\": \"America\/Detroit\",\n      \"tzoffset\": -4.0\n    }\n  }\n}\n<\/code><\/pre>\n\n\n\n
Other chunks of code can take care of sorting through this mess – we just have to make sure the file updates when it needs to.<\/p>\n\n\n\n
OK, Now what?<\/p>\n\n\n\n
In the old do-it-yourself system, CRON scripts ran every 2 minutes to do this. Not all web hosts allow you to run CRON jobs, especially every 2 minutes.  The solution is to have another<\/em> service run a script every 2 minutes, and that script tells our scripts to run. <\/p>\n\n\n\n
Sound complicated?<\/p>\n\n\n\n
Maybe not! This should be a turnkey solution. As long as there’s a scootercam.net capable of receiving a little ping, this will work. Without delay, here’s <\/p>\n\n\n\n
ScooterCam Weather & Moon Phase System<\/h1>\n\n\n\n
A reliable, portable, easy-to-maintain data update system<\/strong><\/p>\n\n\n\n
\n\n\n\n
\ud83d\udce6 What’s in This Package<\/h2>\n\n\n\n
This is a complete cron-free update system<\/strong> for both weather and moon phase data, designed for low-traffic sites with maximum portability and ease-of-use.<\/p>\n\n\n\n
Weather System Files (Updates every 5 minutes)<\/h3>\n\n\n\n
    \n
  • wx-update.php<\/strong> – Secure webhook endpoint for weather updates<\/li>\n\n\n\n
  • wx-status.php<\/strong> – Visual monitoring dashboard (weather only)<\/li>\n\n\n\n
  • api-get-viscross.php<\/strong> – Your existing weather data fetcher (no changes needed)<\/li>\n<\/ul>\n\n\n\n
    Moon Phase System Files (Updates hourly)<\/h3>\n\n\n\n
      \n
    • moon-update.php<\/strong> – Secure webhook endpoint for moon phase updates<\/li>\n\n\n\n
    • api-get-moon.php<\/strong> – Moon phase data fetcher<\/li>\n\n\n\n
    • wx-status-combined.php<\/strong> – Enhanced dashboard showing both weather AND moon data<\/li>\n<\/ul>\n\n\n\n
      Diagnostic Tools<\/h3>\n\n\n\n
        \n
      • wx-install-check.php<\/strong> – One-time installation diagnostic (delete after setup)<\/li>\n<\/ul>\n\n\n\n
        Documentation<\/h3>\n\n\n\n
          \n
        • START-HERE.md<\/strong> – Overview and quick start guide<\/li>\n\n\n\n
        • SETUP-CHECKLIST.md<\/strong> – Step-by-step setup checklist (weather)<\/li>\n\n\n\n
        • MOON-SETUP.md<\/strong> – Step-by-step setup for moon phase system<\/li>\n\n\n\n
        • ARCHITECTURE.md<\/strong> – Visual system diagrams and architecture<\/li>\n\n\n\n
        • README.md<\/strong> – This file<\/li>\n<\/ul>\n\n\n\n
          \n\n\n\n
          \ud83c\udfaf What This Solves<\/h2>\n\n\n\n
          Your Old System (Cron-Based)<\/h3>\n\n\n\n
          Server Cron \u2192 Runs scripts every X minutes\n              \u2193\n         Problems:\n         \u274c Requires SSH\/terminal access\n         \u274c Doesn't work on low-traffic sites (WordPress cron)\n         \u274c Hard to move between hosting providers\n         \u274c Fails silently without notifications\n         \u274c Difficult for non-technical users to manage\n<\/code><\/pre>\n\n\n\n
          Your New System (Webhook-Based)<\/h3>\n\n\n\n
          Cron-Job.org (external) \u2192 Calls your webhook URLs\n                           \u2193\n                   wx-update.php or moon-update.php\n                           \u2193\n                   Runs your data fetchers\n                           \u2193\n                   Updates JSON files\n                           \u2193\n         Benefits:\n         \u2705 Works on ANY hosting provider\n         \u2705 Works on low-traffic sites\n         \u2705 Easy to move between hosts\n         \u2705 Email alerts when failures occur\n         \u2705 Visual monitoring dashboard\n         \u2705 Anyone can manage it\n         \u2705 Completely FREE!\n<\/code><\/pre>\n\n\n\n
          \n\n\n\n
          \ud83d\ude80 Quick Start<\/h2>\n\n\n\n
          Which System Do You Want?<\/h3>\n\n\n\n
          Option 1: Weather Only<\/strong> (5 minutes) \u2192 Follow SETUP-CHECKLIST.md<\/strong><\/p>\n\n\n\n
          Option 2: Weather + Moon Phase<\/strong> (10 minutes) \u2192 Follow SETUP-CHECKLIST.md<\/strong> first, then MOON-SETUP.md<\/strong><\/p>\n\n\n\n
          Option 3: Just Want to Understand Everything<\/strong> \u2192 Read START-HERE.md<\/strong> for complete overview<\/p>\n\n\n\n
          \n\n\n\n
          \ud83d\udcca System Overview<\/h2>\n\n\n\n
          Weather System (Every 5 Minutes)<\/h3>\n\n\n\n
          Cron-Job.org\n    \u2193 (calls every 5 min)\nwx-update.php (verifies token)\n    \u2193\napi-get-viscross.php (fetches from Visual Crossing API)\n    \u2193\n\/wx\/*.json files updated\n    \u2193\nYour WordPress plugins display fresh data\n<\/code><\/pre>\n\n\n\n
          Moon Phase System (Every Hour)<\/h3>\n\n\n\n
          Cron-Job.org\n    \u2193 (calls every hour)\nmoon-update.php (verifies token)\n    \u2193\napi-get-moon.php (fetches from RapidAPI)\n    \u2193\n\/wx\/moon_api_data.json updated\n    \u2193\nYour site displays current moon phase\n<\/code><\/pre>\n\n\n\n
          Both Systems Are:<\/h3>\n\n\n\n
            \n
          • \u2705 Independent (separate tokens, separate schedules)<\/li>\n\n\n\n
          • \u2705 Monitored (email alerts for failures)<\/li>\n\n\n\n
          • \u2705 Logged (activity logs in \/wx\/ directory)<\/li>\n\n\n\n
          • \u2705 Portable (works on any hosting provider)<\/li>\n<\/ul>\n\n\n\n
            \n\n\n\n
            \ud83c\udfa8 Key Features<\/h2>\n\n\n\n
            1. External Monitoring (No Server Cron!)<\/h3>\n\n\n\n
              \n
            • Uses Cron-Job.org<\/strong> (free service, no credit card)<\/li>\n\n\n\n
            • Calls your webhooks at scheduled intervals<\/li>\n\n\n\n
            • Sends email alerts if updates fail<\/li>\n\n\n\n
            • Works on ALL hosting providers (shared, VPS, cloud)<\/li>\n<\/ul>\n\n\n\n
              2. Security<\/h3>\n\n\n\n
                \n
              • Token-based authentication (separate tokens for each system)<\/li>\n\n\n\n
              • Protected endpoints (401\/403 errors for unauthorized access)<\/li>\n\n\n\n
              • Activity logging for audit trail<\/li>\n\n\n\n
              • Easy to rotate credentials<\/li>\n<\/ul>\n\n\n\n
                3. Visual Monitoring<\/h3>\n\n\n\n
                  \n
                • wx-status.php<\/strong> – Weather data dashboard<\/li>\n\n\n\n
                • wx-status-combined.php<\/strong> – Combined weather + moon dashboard<\/li>\n\n\n\n
                • Color-coded health indicators (green\/yellow\/red)<\/li>\n\n\n\n
                • Shows last update times for all files<\/li>\n\n\n\n
                • Mobile-friendly responsive design<\/li>\n<\/ul>\n\n\n\n
                  4. Error Handling & Logging<\/h3>\n\n\n\n
                    \n
                  • Graceful failures (old data remains if API fails)<\/li>\n\n\n\n
                  • Detailed error logging in \/wx\/ directory<\/li>\n\n\n\n
                  • Clear error messages in responses<\/li>\n\n\n\n
                  • Email notifications via Cron-Job.org<\/li>\n<\/ul>\n\n\n\n
                    5. Zero Maintenance<\/h3>\n\n\n\n
                      \n
                    • Set it and forget it<\/li>\n\n\n\n
                    • Automatic updates at scheduled intervals<\/li>\n\n\n\n
                    • Email alerts only when issues occur<\/li>\n\n\n\n
                    • No manual intervention needed<\/li>\n<\/ul>\n\n\n\n
                      \n\n\n\n
                      \ud83d\udccb What You Get After Setup<\/h2>\n\n\n\n
                      Weather System:<\/h3>\n\n\n\n
                        \n
                      • Weather data updates automatically every 5 minutes<\/li>\n\n\n\n
                      • Files created\/updated in \/wx\/<\/code> directory:\n
                          \n
                        • current.json<\/code> – Current conditions<\/li>\n\n\n\n
                        • hourly.json<\/code> – 48-hour forecast<\/li>\n\n\n\n
                        • daily.json<\/code> – 7-day forecast<\/li>\n\n\n\n
                        • alerts.json<\/code> – Weather alerts<\/li>\n\n\n\n
                        • summary.json<\/code> – Compact summary<\/li>\n\n\n\n
                        • weather.json<\/code> – Combined data<\/li>\n\n\n\n
                        • weather_table.php<\/code> – HTML table<\/li>\n\n\n\n
                        • raw_weather.json<\/code> – API backup<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n
                          Moon Phase System (Optional):<\/h3>\n\n\n\n
                            \n
                          • Moon phase data updates automatically every hour<\/li>\n\n\n\n
                          • File created\/updated: \/wx\/moon_api_data.json<\/code><\/li>\n<\/ul>\n\n\n\n
                            Monitoring:<\/h3>\n\n\n\n
                              \n
                            • Status dashboard at: scootercam.net\/wx-status.php<\/code> (or wx-status-combined.php)<\/li>\n\n\n\n
                            • Activity logs: \/wx\/webhook.log<\/code> and \/wx\/moon-webhook.log<\/code><\/li>\n\n\n\n
                            • Email alerts from Cron-Job.org if failures occur<\/li>\n<\/ul>\n\n\n\n
                              \n\n\n\n
                              \ud83d\udd27 Technical Details<\/h2>\n\n\n\n
                              System Requirements<\/h3>\n\n\n\n
                                \n
                              • PHP:<\/strong> 7.0 or higher (you already have this)<\/li>\n\n\n\n
                              • Web hosting:<\/strong> Any provider with file access (FTP\/cPanel)<\/li>\n\n\n\n
                              • Server access needed:<\/strong> None! (that’s the point!)<\/li>\n\n\n\n
                              • WordPress required:<\/strong> No (works with or without)<\/li>\n\n\n\n
                              • Database required:<\/strong> No<\/li>\n<\/ul>\n\n\n\n
                                API Services Used<\/h3>\n\n\n\n
                                  \n
                                • Visual Crossing Weather API<\/strong> – Weather data (1000 free calls\/day)<\/li>\n\n\n\n
                                • RapidAPI Moon Phase<\/strong> – Moon phase data (your existing subscription)<\/li>\n\n\n\n
                                • Cron-Job.org<\/strong> – Scheduling service (free tier is sufficient)<\/li>\n<\/ul>\n\n\n\n
                                  File Paths<\/h3>\n\n\n\n
                                  All files should be in your website root (public_html<\/code>):<\/p>\n\n\n\n
                                  public_html\/\n\u251c\u2500\u2500 api-get-viscross.php    (your existing weather script)\n\u251c\u2500\u2500 api-get-moon.php        (your existing moon script)\n\u251c\u2500\u2500 wx-update.php           (NEW - weather webhook)\n\u251c\u2500\u2500 moon-update.php         (NEW - moon webhook)\n\u251c\u2500\u2500 wx-status.php           (NEW - status dashboard)\n\u251c\u2500\u2500 wx-status-combined.php  (NEW - combined dashboard)\n\u2514\u2500\u2500 wx\/                     (directory for data files)\n    \u251c\u2500\u2500 current.json\n    \u251c\u2500\u2500 hourly.json\n    \u251c\u2500\u2500 daily.json\n    \u251c\u2500\u2500 alerts.json\n    \u251c\u2500\u2500 summary.json\n    \u251c\u2500\u2500 weather.json\n    \u251c\u2500\u2500 weather_table.php\n    \u251c\u2500\u2500 raw_weather.json\n    \u251c\u2500\u2500 moon_api_data.json\n    \u251c\u2500\u2500 webhook.log\n    \u2514\u2500\u2500 moon-webhook.log\n<\/code><\/pre>\n\n\n\n
                                  \n\n\n\n
                                  \ud83d\udd10 Security<\/h2>\n\n\n\n
                                  Two Independent Systems = Two Separate Tokens<\/h3>\n\n\n\n
                                  Weather Token<\/strong> (for wx-update.php):<\/p>\n\n\n\n
                                    \n
                                  • Used to call weather webhook<\/li>\n\n\n\n
                                  • Should be different from moon token<\/li>\n\n\n\n
                                  • Change every 6 months<\/li>\n<\/ul>\n\n\n\n
                                    Moon Token<\/strong> (for moon-update.php):<\/p>\n\n\n\n
                                      \n
                                    • Used to call moon phase webhook<\/li>\n\n\n\n
                                    • Should be different from weather token<\/li>\n\n\n\n
                                    • Change every 6 months<\/li>\n<\/ul>\n\n\n\n
                                      Security Best Practices<\/h3>\n\n\n\n
                                        \n
                                      • \u2705 Use strong random tokens (20+ characters)<\/li>\n\n\n\n
                                      • \u2705 Never commit tokens to version control<\/li>\n\n\n\n
                                      • \u2705 Store tokens in password manager<\/li>\n\n\n\n
                                      • \u2705 Don’t share tokens in support requests<\/li>\n\n\n\n
                                      • \u2705 Delete wx-install-check.php after setup<\/li>\n\n\n\n
                                      • \u2705 Monitor \/wx\/*.log files for suspicious activity<\/li>\n<\/ul>\n\n\n\n
                                        \n\n\n\n
                                        \ud83d\udcc8 Update Schedules<\/h2>\n\n\n\n
                                        Weather: Every 5 Minutes<\/h3>\n\n\n\n
                                          \n
                                        • Why?<\/strong> Weather changes frequently, especially during storms<\/li>\n\n\n\n
                                        • API usage:<\/strong> ~8,640 calls\/month (well under 30,000\/month limit)<\/li>\n\n\n\n
                                        • Data freshness:<\/strong> Always current<\/li>\n<\/ul>\n\n\n\n
                                          Moon Phase: Every Hour<\/h3>\n\n\n\n
                                            \n
                                          • Why?<\/strong> Moon phase changes very slowly<\/li>\n\n\n\n
                                          • API usage:<\/strong> ~720 calls\/month<\/li>\n\n\n\n
                                          • Data freshness:<\/strong> More than sufficient<\/li>\n<\/ul>\n\n\n\n
                                            Can I Change These?<\/h3>\n\n\n\n
                                            Yes!<\/strong> Edit your Cron-Job.org schedules:<\/p>\n\n\n\n
                                              \n
                                            • Weather: Can go up to every 1 minute (but wastes API calls)<\/li>\n\n\n\n
                                            • Moon: Could update every 6-12 hours (moon barely changes)<\/li>\n<\/ul>\n\n\n\n
                                              \n\n\n\n
                                              \ud83c\udd98 Troubleshooting Quick Reference<\/h2>\n\n\n\n
                                              Weather Not Updating?<\/h3>\n\n\n\n
                                                \n
                                              1. Check scootercam.net\/wx-status.php<\/code> – Are files stale?<\/li>\n\n\n\n
                                              2. Check Cron-Job.org dashboard – Is job enabled and running?<\/li>\n\n\n\n
                                              3. Test manually: scootercam.net\/wx-update.php?token=YOUR_TOKEN<\/code><\/li>\n\n\n\n
                                              4. Check \/wx\/webhook.log<\/code> for error messages<\/li>\n<\/ol>\n\n\n\n
                                                Moon Phase Not Updating?<\/h3>\n\n\n\n
                                                  \n
                                                1. Check scootercam.net\/wx-status-combined.php<\/code> – Is moon data stale?<\/li>\n\n\n\n
                                                2. Check Cron-Job.org dashboard – Is moon job enabled?<\/li>\n\n\n\n
                                                3. Test manually: scootercam.net\/moon-update.php?token=YOUR_MOON_TOKEN<\/code><\/li>\n\n\n\n
                                                4. Check \/wx\/moon-webhook.log<\/code> for error messages<\/li>\n<\/ol>\n\n\n\n
                                                  “Unauthorized” or “Forbidden” Errors?<\/h3>\n\n\n\n
                                                    \n
                                                  • Token mismatch – verify token in PHP file matches token in URL<\/li>\n\n\n\n
                                                  • Check for extra spaces or quotes in token<\/li>\n<\/ul>\n\n\n\n
                                                    Still Stuck?<\/h3>\n\n\n\n
                                                      \n
                                                    • Read ARCHITECTURE.md<\/strong> for system diagrams<\/li>\n\n\n\n
                                                    • Check Cron-Job.org execution history<\/li>\n\n\n\n
                                                    • Verify file permissions on \/wx\/<\/code> directory (should be writable)<\/li>\n\n\n\n
                                                    • Contact your hosting provider if file write issues persist<\/li>\n<\/ul>\n\n\n\n
                                                      \n\n\n\n
                                                      \ud83d\ude9a Migration to New Hosting<\/h2>\n\n\n\n
                                                      Moving to a new provider? Easy!<\/strong><\/p>\n\n\n\n
                                                      What to Do:<\/h3>\n\n\n\n
                                                        \n
                                                      1. Download<\/strong> all files from old host:\n
                                                          \n
                                                        • All .php files<\/li>\n\n\n\n
                                                        • Everything in \/wx\/ directory<\/li>\n<\/ul>\n<\/li>\n\n\n\n
                                                        • Upload<\/strong> to new host (same directory structure)<\/li>\n\n\n\n
                                                        • Update Cron-Job.org<\/strong> URLs:\n