I find that every project I work on I tend to steer the code in a different direction. Often this is a response to the project. With Python, I have this flexibility – at times I focus on functions, other times I’ll give data the bulk of my attention.

I used to write a lot of small Bash and Python interactions, but more and more my experience is trending toward larger projects that involve constructing an entire back-end using these languages. Either way – over the years I have written some neat code (I think).

Auto-reestablish API session

I encountered a service with an inconvenient combination of problems: it was slow to get all data, process it, and submit responses back incrementally. I couldn’t batch the problem, but this all left the interaction vulnerable to the session token timing out.

The process to resolve this involved implementing Python dataclasses to manage both the connection details and API session token. This allowed the data to persist and be efficiently checked before every API interaction. When the expiretime was reached, a process to _reestablish the token was kicked off.

def _reestablish(self):
    Use this to peroidically re-establish your 
    API Key. During long running API queries
    you'll end up with an exipred token and receive
    401 errors.
    This check is triggered to handle those.

    if self.expiretime < round(time.time()):
            f"API token session has expired. Expiration time: {datetime.fromtimestamp(self.expiretime).strftime('%c')}")
        object.__setattr__(self, 'token', _token(test=self.test))
        object.__setattr__(self, 'expiretime', round(time.time()) + self.duration)
        object.__setattr__(self, 'headers', headers)

I’d never worked with manipulating the loaded object object in Python, but a dataclass can be manipulated with values from any function that has access to the object. I don’t know how frequent I will use this manipulation, but it was certainly useful here!

State Checks

When I’m integrating data from several sources, there’s often an ugly combination of logic that will become unavoidable at some point. It’s along the lines of:

if ( A in set1 and setB ) and not (if A.1 exists and ( A.1 == 2 )):
    . . .


I had some really problematic code that was causing issues and went looking for design patterns in Python that would help to manage just this type of logic. I found a clever feature of Python: all().

Here’s a readable chunk of code from the linked site:

>>> bool_exps = [
...     5 > 2,
...     1 == 1,
...     42 < 50,
... ]
>>> all_true(bool_exps)

My real world implementation is a little more…involved. I’m sure there’s performance advantages, but it’s also really nice to be able to print a dictionary in debug logging that would just show why the conditions aren’t met. I think I’ll use this for to create better auditing/reporting during evaluation processes.

if item in list(
    state_check = {
        "item_id_exists": True,
        "item_unused": True 
            if[item] not in comparision_dict
            else False,
    state_check = {"item_id_exists": False}

if all(list(state_check.values())):"Beginning delete process for {item}: {state_check}")

Send HTML emails in Bash

mail can easily submit an email to a configure relay or mail server and provide some basic features. Presentation however…that requires some work. There are clear advantages to plain text, but anymore I really prefer to have a nice looking list/report that I can easily review. I don’t use emails for monitoring as much as I used to – instead it’s typically reporting – which should look nice!

In Bash you can submit mail direct to the sendmail program. It takes a formatted email message, parses, and handles it.

    echo -e From:
    echo -e To:
    echo -e Subject: Long running jobs
    echo -e Content-Type: text/html
    echo -e "
        The following jobs took longer this week:

            echo "${results[@]}" | sed 's/ /\n/g' | column -N "JOB,ID,START DATE, EXPECTED RUNTIME, ACTUAL RUNTIME" -t -s','

        [$(hostname -f)] - $(date)
) | /usr/sbin/sendmail -t

Would produce a message like this:

The following jobs took longer this week:

FeedMe 24  2023-09-08T11:35    12s             94s

[] - Fri Sep 8 17:51:14 CDT 2023

If I tried to send a plain formatted table in Bash, I can’t control the plain text table – the email viewer will simply output the text in the system font:

The following jobs took longer this week:

FeedMe 24 2023-09-08T11:35 12s 94s

[] - Thu Nov 2 11:32:37 CDT 2023

This pattern is easily reproducible and looks nice in scripts. It means I can simply drop my variables and subshells directly into the HTML layout. In the Bash world, I consider that a big win.

The one major detraction here: you cannot send attachments (well…not reasonably at least IMHO) using this method

Shell Parameter Expansions

I love shell parameter expansions. I think of it like the regex of Bash variables. It’s ugly, relatively unreadable, but still worth it.

Let’s look at my favorites:


If you suspect you might collect a null variable that you don’t want, Bash can allow you set a default:

$ var=
$ : ${var:=DEFAULT}
$ echo $var

Pattern Matching

I use this a lot to manage subdirectories and dynamically specify either a file or it’s containing directory without having multiple variables or using awk to parse out components.

$ data_location='data/folderA/mydata.txt'
# Directories:
$ echo -e ${data_location%/**}
# File
$ echo -e ${data_location##*/}


Frequently, you’ll get a resulting variable from an external source or something internally that needs to dynamically update. This means you’ll need to update the value within a variable. Rather than overwriting the variable, parameter expansion can replace components:

$ data_location='data/folderA/mydata.txt'
$ echo -e ${data_location/my/your}


I’m sure there’s more, but these just hit me right today.