YDMmobile / oss

Fourteen Green Runs and Ten Days of Silence: Your Scheduler Runs a Different Bash

2026-08-03

I have a small script that writes a daily journal entry. It reads what changed across my repos, summarizes it, and drops a Markdown file into a folder. Nothing clever. It had passed shellcheck with no warnings and fourteen end-to-end runs on my own machine.

It then failed every single day for ten days, and I did not notice.

The interesting part is not that it broke. The interesting part is why nothing told me — and the answer turns out to be a general lesson about scheduled jobs that I had been getting wrong for years.

The part where everything looks fine

Here is the shape of the problem, and it is a nasty one: the output folder looked complete.

If you had opened my journal directory on day nine of the outage, you would have seen a tidy run of dated files with no gaps. No missing dates. Nothing to notice. The one signal a human actually checks — is the thing it produces there? — was clean, because I would occasionally backfill by hand without registering that I was papering over a systematic failure rather than catching up on a lazy week.

The failure only becomes visible if you look at file modification times instead of file names:

2026-07-17  mtime=2026-07-27   <-- written 10 days late
2026-07-18  mtime=2026-07-27   <-- written  9 days late
2026-07-19  mtime=2026-07-27   <-- written  8 days late
2026-07-20  mtime=2026-07-27   <-- written  7 days late
...
2026-07-26  mtime=2026-07-27

Ten files, ten different dates, all written in one burst on the 27th. That is not a schedule. That is a person cleaning up after something that stopped working on the 17th.

The part where the logs lie by omission

My script logs. It logs when it starts, when it finds nothing to do, when it writes a file. During the outage the application log gained zero new lines.

The obvious reading of "the log is empty" is "the job never ran." That reading is wrong, and it is the reason the outage lasted as long as it did. The job ran. It ran on time, every day, and died before reaching its first logging statement — so the log had nothing to say about it.

This is a specific diagnostic signature worth memorizing, because it points somewhere very different from "the scheduler is broken":

The scheduler reports runs with a nonzero exit status, while the application's own log has zero new lines.

That combination means the process was started and then failed before it got far enough to tell you anything. Your application log is the wrong place to look. The evidence is in whatever the scheduler captured, which is a different file that most of us configure once and never read again.

Mine had been dutifully recording the answer the whole time:

/Users/…/daily-journal.sh: line 73: unexpected EOF while looking for matching `''
/Users/…/daily-journal.sh: line 73: unexpected EOF while looking for matching `''
/Users/…/daily-journal.sh: line 73: unexpected EOF while looking for matching `''

Ten identical lines. One per day. Sitting in a file I had not opened since the day I set the job up.

The part where I learn which Bash I am actually running

unexpected EOF on a script that passes shellcheck is a contradiction, and the contradiction is the clue. Two parsers disagreed, which means two parsers were involved.

Here is the plist that runs the job:

<key>ProgramArguments</key>
<array>
    <string>/bin/bash</string>
    <string>/Users/…/.claude/scripts/daily-journal.sh</string>
</array>

/bin/bash. Not bash. Not /usr/bin/env bash. The literal path.

On macOS, that path is not the Bash you have been testing with:

$ /bin/bash --version
GNU bash, version 3.2.57(1)-release

$ bash --version
GNU bash, version 5.3.15(1)-release

Apple has shipped that same Bash at /bin/bash for a very long time — the copyright line in its own --version output reads 2007. My interactive shell resolves bash through Homebrew to 5.3. Every one of my fourteen "end-to-end" runs used 5.3. launchd used 3.2. I had verified the script against an interpreter that never executes it.

The construct at line 73 was a heredoc inside command substitution whose body contained an apostrophe. You can reproduce the disagreement in about ten seconds:

cat > probe.sh <<'PROBE'
X=$(cat <<EOF
it's a test
EOF
)
echo "ok: $X"
PROBE

/bin/bash -n probe.sh   # syntax error: unexpected EOF while looking for matching `'
bash -n probe.sh        # silent — parses fine

The 3.2 parser loses track of quoting when a heredoc is nested inside $( ) and the body contains an unpaired apostrophe. The 5.x parser does not.

I want to be careful here, because it is tempting to walk away with a bigger rule than the evidence supports. This is one construct failing under one interpreter. It is not a claim that every heredoc breaks on Bash 3.2, and I have not built a version matrix across patch releases and delimiter forms. If you hit something adjacent, reproduce it against your actual deployment interpreter rather than trusting my anecdote — which is, after all, the entire point of the article.

The fix, and the smaller fix that matters more

The code fix is unglamorous. Assign the multiline content without nesting a heredoc inside command substitution:

read -r -d '' VAR <<EOF || true
…multiline content…
EOF

The || true is there because read returns nonzero when it hits EOF without a delimiter, which is exactly what happens here and is not an error.

But the code fix is not the lesson. I could write that construct again tomorrow in a different script and be equally blind. The lesson is that "I tested it" was never true — I tested a lookalike.

So the check I actually added is a sequence, and it is short:

  1. Read the literal interpreter out of the plist, crontab, or hook config. Do not resolve it through PATH; PATH is what lied to you.
  2. Run that interpreter's syntax check: /bin/bash -n script.sh.
  3. Fire the real trigger — for launchd, launchctl kickstart, not bash script.sh.
  4. Look at the artifact it was supposed to produce.
  5. Ask the scheduler for the exit status.

Step 5 on my machine now says:

$ launchctl print gui/$UID/kr.donminzzi.claude-daily-journal | grep -i "runs\|last exit"
	runs = 4
	last exit code = 0

(That field name is from my machine today rather than from documentation I can point you at — check what your own launchctl print calls it before scripting against it.)

Steps 1 and 2 alone would have caught this in the two seconds it takes to type them.

What I actually changed my mind about

I used to treat "it works on my machine" as a joke about dependency versions and different operating systems. It is also a joke about the same machine. My laptop contains at least two Bash interpreters, and which one runs my code depends entirely on who invokes it — me, or launchd, or a git hook, or CI. Interactive success says nothing about scheduled success, because they are not the same execution.

Three things I now believe that I did not on the 17th:

  • A green test run proves the path you exercised, and only that path. Fourteen runs under the wrong interpreter is not fourteen pieces of evidence. It is one piece of evidence, repeated.
  • An empty application log during a failure is a routing signal, not a mystery. It means look at the scheduler's captured stderr, not at your logging code.
  • The existence of the artifact is not proof the job produced it. Check timestamps, not filenames. A folder with no gaps can be a folder someone has been quietly repairing.

If you have a scheduled job you have not looked at in a while, the cheapest possible version of this article is one command. Open the plist or crontab, find the literal interpreter, and run its syntax check against your script. It takes ten seconds and it either tells you nothing or it tells you something you very much wanted to know ten days ago.