Michael Sendker

What I Can Do

I prefer Python, but I can wear a lot of hats. I have written code in Python, JavaScript, ReactJS, and C#. I have dabbled in PHP, C, C++, Scheme, and Bash. I have built static sites in GatsbyJS, interfaces in WPF, and webscrapers in Python. I've done data analysis and visualization with Pandas and Seaborn. I've messed with Scikit-Lean. I've done generative art with PyCairo, of which, the picture above is a sample. I even built an Arduino-based PID controller for a popcorn popper to roast my own coffee.

I am also a rigorous documenter. I often make sure I have my documentation system ready to go as I write the software. You can see an example of my documentation-in-progress for ICC2 here. And I am meticulous about my system management and organization. I use YADM to manage my dotfiles in a very particular way (see .zshrc and.zshrc.d/ for an example). I'm still surprised I've never seen anyone do that before.

Below is a semi-exhaustive list of technology tasks I've performed

CRUD web applications
Single page web applications
Static portfolio sites (both single page Gatsby-based and HTML-based)
Twitter bots
Web/Api scraping
Data cleaning
Data visualization
Data analysis
Arduino-based hardware and circuitry
Server deployment/management
Python testing
Desktop application interfaces
ETL scripts
Text processing
Complex git structures (submodules)
Linux ricing
Open Source contribution
Twilio-based texting notification bots
Multithreading
Task automation / general purpose scripting
Encryption using GPG, Keybase, and Matrix.org
Graphics work with Photoshop and GIMP

Below is an exhaustive list of languages I've worked with sorted by strength

Python
JavaScript
C#
SQL
HTML/CSS
C
Bash
Arduino-based C++
Scheme
PHP

Below is a semi-exhaustive list of technologies I've worked with

Web Development: Flask, Django, ReactJS, GatsbyJS, Sergey, Vanilla JS, JQuery, Sass, CSS3, Styled-Components, Auth0
Data management: SQLAlchemy, Django ORM
Data Storage: MariaDB, MySQL, PostgreSQL, DynamoDB, SQLite, AWS DynamoDB, ElasticSearch
Web Scraping: Requests, urllib3, BeautifulSoup4, Selenium
Data Analysis: Jupyter Notebooks, pandas, NumPy, Matplotlib, Seaborn, Scikit-Learn
Cloud Applications: Heroku, AWS EC2, AWS Lambda, General VPS
Linux: Arch Linux, Ubuntu, Debian, Gnome, i3wm, BSPWM/SXHKD, Cinnamon, KDE
Web Design: Styled Components, Sass, CSS3, Photoshop, GIMP
Miscellaneous: Docker, Docker Compose, Excel, PyCairo, gspread, openpyxl, Systemd Units, Chron

What I've Done

Intertextual Canon Cloud (anno.wiki)

The ICC is a web application designed to allow for collaboratively building an exhaustive and definitive repository of annotated literature. I designed, developed, deployed, and continue to maintain the project solo, managing occasionally to rope in some programmer friends for help with various features.It consists of ≈14k lines of code and ≈100k lines of code churn. The backend uses Flask/SQLAlchemy. The frontend uses Jinja2, Sass, and VanillaJS. I also had to write several ETL data pipelines for processing Project Gutenberg texts. It is deployed via Heroku. I learned the entire web application life cycle on this project, and it continues to teach me. It is currently maintained at https://anno.wiki. I am also currently working on a second iteration of the site.

Primarily coded in Flask/SQLAlchemy
Wrote many ETL data pipeline scripts for processing Project Gutenberg text files into custom JSON formats
Heroku for platform
MariaDB for database
Elasticsearch for search
Vanilla JavaScript for AJAX and progressive enhancements
Talisman for CSRF security
pytest for testing
WTForms for forms

Euler Sci Website

This was a pretty simple project but turned out so beautiful I love it. It was also one of my first paid projects and helped me acquire the contract for working on an interface for a laser system.

Static web site designed in GatsbyJS
Graphics work done in photoshop
Color palette designed through coolors.co

Amazon BuyBox Notifier

This project was for a client who sells various vitamins on Amazon. I developed, and redeveloped over several successive weeks, a rather large web scraping/api scraping system that gleans information about Amazon Marketplace Offers on products the client is selling from multiple data sources.

Because of the increasing complexity of the project it ended up becoming rather large, with a decent amount of features:

multithreading capabilities
a token tracking system to make sure to stay within the rate limit
header spoofing and proxy rotation
near real time updates for various settings including which ASINS to track and notification endpoints using cloud file storage
a PostgreSQL database for tracking data (using SQLAlchemy as the ORM, of course).
database synchronization with an Elasticsearch instance for a Kibana dashboard

The software was deployed to an Arch Linux VPS and used a Systemd Unit to run. I've done automated data gathering (mostly api/web scraping) before, but never on this level. I had a lot of fun, building from the ground up. I'm especially glad for having learned to do multithreading at a decent level.

Language Map

This project was inspired by a trend on r/languagelearning where people would post world maps with the countries colored by the languages they spoke. I decided to build an actual dedicated web app that would automatically color the countries themselves.

To do this required a lot of data gathering and cleaning. Doing that itself was an arduous process that had a lot of false starts. The final method I settled on was to use Google Sheets importhtml function, which could be used to import Wikipedia tables to a spreadsheet. To automate this I used Python and gspread.

It was definitely a lesson in data gathering and cleaning.

Extensive data gathering and web scraping mission detailed here
BeautifulSoup4, requests, PubProxy, scrapestack
gspread, Google Sheets API
pandas, csv, json
ReactJS
React Simple Maps
Material-UI
Hosted on Netlify

Roastuino

As part of my continuing obsession with coffee I discovered that I could get a lot cheaper specialty coffee, roasted closer to my personal taste, from the regions I prefer (East Africa), by just roasting myself. Sweet Maria's provides troves of information about how to home roast, and one of the suggestions they make for first timers is to use an inexpensive popcorn popper. They recommend the Nostalgia.

So I bought it, and was dissatisfied with the lack of control I had. I found out that there are a not insignificant number of home roasters who tried hooking up a PID controller to their popcorn popper (or generally modding it), and I found this decent write up on how to use an Arduino to hook up the heating element to a 40A relay and use a thermocouple with an RTU Modbus implementation to communicate with Artisan Scope, a professional grade open source program written in Python for monitoring and controlling roast parameters (more meant for large machines, but...)

There are some problems with Lukas' write up. Namely, that Artisan's internal PID system is really hard to get working. I ended up building in an internal PID using an Arduino library (you can see my code here). So Artisan just sends the PID values and the SV and monitors the temperature. I also found it unnecessary to use a Mosfet between the Arduino and the relay. And for debugging, I added an LCD screen (which is how I discovered that Artisan's PID was never working in the first place).

This project was insane fun. As a programmer, I often get a kick out of some piece of software working. But making something do stuff in meatspace is a rush I'd never experienced before. It was also the first time I'd done real soldering. It was a lot easier than I expected.

Arduino Uno R3 from Elegoo
MAX31855 Thermocouple Amplifier breakout board from Adafruit with a K-type thermocouple
HD44780 LCD
Inkbird 40DA SSR
Artisan Scope
Various Arduino libraries

Duvall Invitational Realtime Scoring Site

My father hosts a golf tournament every year known as the DuVall Invitational. It's been running for over fifty years and is mostly played by men in the construction industry and their friends. But as with everything, using the web allowed them to improve the general quality of life in the tournament.

They use a master Excel spreadsheet to keep track of scoring, auction data, and handicaps that is carried over from year to year. A year or two ago I made some massive improvements to the software infrastructure they use.

First, I remade their spreadsheet from scratch, as it had become a crufty mess of tangled code, with IF(IF(IF())) style formulas that stretched over a hundred characters long with excruciatingly difficult logic. I upgraded them to using the latest version of Excel which allowed me to program it using modern Dynamic Arrays and Named Ranges. This made drop down selection a lot easier, so that the list of options in a given cell could be checked against the sheet itself to filter out used options.

I also created a standalone handicap spreadsheet that allows them to dynamically calculate handicaps by weighting scores from the previous 5 years without having to mess with moving columns and rewriting formulas.

Then I found the lightest-weight static site generator I could, Sergey, to generate a static site that used a little bit of JavaScript to make keeping track of the API keys for the spreadsheet stored in Microsoft OneDrive easier. The result is that my father only has to make changes to a spreadsheet that he uploads to OneDrive and those changes will dynamically update on the webpage in embedded frames that point to specific ranges on that spreadsheet. They find it incredibly helpful to check up on how their competition is doing while out on the course using their smartphones.

I even made a PR upstream to Sergey when I found that the development server automatically exposes to the network using 0.0.0.0. Unfortunately, the maintainer still hasn't merged it. Luckily, it's lightweight enough that I can maintain the SSG myself without worrying about it being no longer maintained.

All the code for the site can be found at malan88/duvall.

JavaScript
HTML
Sass
Sergey
Excel
Microsoft OneDrive

Letters of John and Abigail Adams Twitterbot

This project was inspired by a photo that made the rounds of Twitter in the wake of the protests in June 2020. The photo was of John Trumbull's famous painting of the signing of the Declaration of Independence with red dots over the faces of all the slave-owning founding fathers. Only 8 or 9 of the several dozen men in the photo could boast never having owned slaves and John Adams was smack dab in the middle with his face as apparent as ever.

I've always been a bit of a John Adams fanboy so I decided to put my Python to work making a twitterbot that would tweet lines from the letters of John and Abigail Adams, a lovely repository of romantic correspondence.

The bot tweets out fairly regularized sentences in threads that correspond to each letter in chronological order, and then repeats. I used tweepy for the Twitter interfacing, AWS's Lambda to run the program, and AWS's DynamoDB to keep track of where the bot is. The actual letters are hosted on a static site of json files on Netlify. That simplified some of the deployment for various reasons I can't recall now.

Twitter handle @john_and_abbie
Bot to tweet the corpus of letters between John and Abigail Adams
Data scraped from NARA's The Adams Papers using requests and BeautifulSoup4
Built using Tweepy
Hosted as a lambda function on AWS Lambda
Use AWS DynamoDB to track place in corpus
Use Netlify to host the letters as static text files
urllib3 to access the static text files
Boto3 to access the DynamoDB tracking

Generated Anki decks

Learning French from English is sometimes tricky. Especially with recognizing the difference between certain words, like au-dessous vs au-dessus (below vs above). So I decided to generate an Anki deck with audio files pulled from forvo.com's API that are distinguishable by a single phoneme (the ones I find myself distinguishing). To read my process, please see the README in the Github repository.

The other deck, the numbers deck, consists of 2500 audio files generated through IBM Cloud's text to speech generator. It is the first 2500 numbers in French with their corresponding words and numerical representation. It is sometimes hard to understand French numbers because it uses a mixed decima/vigesimal system (seventy is sixty-and-ten, eighty is four-twenties, and ninety is four-twenties-and-ten).

These projects are simple, but I am proud of them. I seem to have helped a lot of people learning French with them.

The minimal pairs deck, in particular, required a lot of strategizing for dealing with the forvo.com api rate limits (500 requests per day) and the combinatorial explosion of comparing minimal pairs.

Generated an Anki deck for recognizing phonemes difficult to hear for an American English speaker using “minimal pairs,” words in French which differ by only one phoneme using the forvo.com api. Almost 600 downloads and 3 positive reviews.
Also generated an Anki deck for training the ear to hear difficult French numbers (which use a mixed vigesimal, or base-20, system) using IBM Cloud's Text to Speech renderer and the num2words Python library. Over 500 downloads and 3 positive reviews.
Used Requests for automated downloading of the audio files
Used BeautifulSoup4, requests for scraping the list of minimal pairs in an IPython session

standingwater.io & blog.standingwater.io

I developed this portfolio site and the accompanying blog using GatsbyJS, which I've come to increasingly love for quick static site generation. The GraphQL is, while not the most intuitive, at least pretty useful. Especially the dynamic image processing.

The main portfolio site is built from a modified HTML5 Up template called Dimension, that was ported to Gatsby as gatsby-dimension-starter by codebushi. I have modularized it a bit, like this projects section that uses markdown files and images for each project and an array for ordering, inclusion (there are some projects I don't display for NDA reasons, unfortunately).

HTML5 Up makes some beautiful templates, but their code is often way too tightly coupled and difficult to work with.

The blog I built from the default starter.

The style is inspired by my childhood home in Tampa, Florida. I initially wanted to use a background of palmettos, but it proved too visually difficult a background. I went with a swamp, instead, of which we have many. The site name is a reference to a joke name we gave to the 5 1/4 acre "estate" I grew up on. It puddled quite a lot. Almost looked like a lake.

I also used The Noun Project to purchase the heron icon used in both sites as representative of the area. I'd prefer an Ibis, since they're lousy in the area, but a good one was hard to find.

glendalepainting.com

This site was built for my father's commercial painting company. It was my first project in GatsbyJS and was meant to replace glendalepainting.com. Unfortunately, my father is rather slow to update things and he's still using the original Joomla-based site.

The gallery is perhaps the most impressive aspect. A friend of mine helped build that section.

The main page, using react-spring for the parallax, was quite a nightmare. Parallax is not the easiest to design, but it's a dumpster fire when you have to do it for mobile devices using background images instead of sprites.

Never again. Just don't.

GatsbyJS static site
react-spring for parallax animation
react-reveal for transition animations
Deployed using Netlify

vote.py

A friend of mine had never voted, nor registered to vote. So I made sure she did in 2020 by writing this little script to text her using Twilio to remind her to vote on an increasingly frequent basis as the deadline for registration drew near. She registered pretty quick, actually and it didn't run for that long. Which is lucky, because I ended up discovering a breaking bug on the month roll over.

I ended up having to resort to good old fashioned wheedling and emotional manipulation to get her to actually vote, because she ended up just blocking the number.

Automated Python script to encourage and remind people to register to vote.
Deployed using AWS EC2 instance to enable continuous service.
Integrated Twilio as texting client.
Dynamic scheduling using datetime and sched

gobble.py

This was a one off quick project as a joke idea. It's a python script using the secrets module from the Standard Library to generate a password including an English word from the Electronic Freedom Foundation's Diceware list. It produces passwords that look like this: hydrant_34l;k1qjfasdf.

The idea was to circumvent the "gobbledy-gook" social engineering vulnerability in Security Questions. Some people just put a randomly generated string for the security, but then the worry is that when an operator on a phone asks for it, they'd just accept the answer "It's just a bunch of gobbledy-gook". This script's idea is that if there's a single recognizable word in the security phrase, the operator would hopefully at least require that.

A better solution would be to just use diceware, but where's the fun in that?

teamup.py

This script came about because of a family tradition of playing Christmas Trivia. The idea was to create a computerized method for achieving the closest balanced teams of arbitrary size and number.

It takes a JSON file with team members and their skill level (this could be any numeric scale, like 1-10, 1-100, whatever), and then uses a generator to get the best match up over time. That is to say, it looks for the next best teams continuously. There's a huge diminishing return on calculating combinations, and it's extremely computationally expensive to calculate all the possible team combinations, but after about five to ten minutes you get pretty good team arrangements, better than if you just did it by hand.

Michael Sendker

What I Can Do

What I've Done

Who I Am

Contact