Michael Sendker

Full stack web developer. Python generalist. Specialized in Python and React web development.

What I Can Do

I prefer Python, but I can wear a lot of hats. I have written code in Python, JavaScript, ReactJS, and C#. I have dabbled in PHP, C, C++, Scheme, and Bash. I have built static sites in GatsbyJS, interfaces in WPF, and webscrapers in Python. I've done data analysis and visualization with Pandas and Seaborn. I've messed with Scikit-Lean. I've done generative art with PyCairo, of which, the picture above is a sample. I even built an Arduino-based PID controller for a popcorn popper to roast my own coffee.

I am also a rigorous documenter. I often make sure I have my documentation system ready to go as I write the software. You can see an example of my documentation-in-progress for ICC2 here. And I am meticulous about my system management and organization. I use YADM to manage my dotfiles in a very particular way (see .zshrc and.zshrc.d/ for an example). I'm still surprised I've never seen anyone do that before.

Below is a semi-exhaustive list of technology tasks I've performed

  • CRUD web applications
  • Single page web applications
  • Static portfolio sites (both single page Gatsby-based and HTML-based)
  • Twitter bots
  • Web/Api scraping
  • Data cleaning
  • Data visualization
  • Data analysis
  • Arduino-based hardware and circuitry
  • Server deployment/management
  • Python testing
  • Desktop application interfaces
  • ETL scripts
  • Text processing
  • Complex git structures (submodules)
  • Linux ricing
  • Open Source contribution
  • Twilio-based texting notification bots
  • Multithreading
  • Task automation / general purpose scripting
  • Encryption using GPG, Keybase, and Matrix.org
  • Graphics work with Photoshop and GIMP

Below is an exhaustive list of languages I've worked with sorted by strength

  • Python
  • JavaScript
  • C#
  • SQL
  • HTML/CSS
  • C
  • Bash
  • Arduino-based C++
  • Scheme
  • PHP

Below is a semi-exhaustive list of technologies I've worked with

  • Web Development: Flask, Django, ReactJS, GatsbyJS, Sergey, Vanilla JS, JQuery, Sass, CSS3, Styled-Components, Auth0
  • Data management: SQLAlchemy, Django ORM
  • Data Storage: MariaDB, MySQL, PostgreSQL, DynamoDB, SQLite, AWS DynamoDB, ElasticSearch
  • Web Scraping: Requests, urllib3, BeautifulSoup4, Selenium
  • Data Analysis: Jupyter Notebooks, pandas, NumPy, Matplotlib, Seaborn, Scikit-Learn
  • Cloud Applications: Heroku, AWS EC2, AWS Lambda, General VPS
  • Linux: Arch Linux, Ubuntu, Debian, Gnome, i3wm, BSPWM/SXHKD, Cinnamon, KDE
  • Web Design: Styled Components, Sass, CSS3, Photoshop, GIMP
  • Miscellaneous: Docker, Docker Compose, Excel, PyCairo, gspread, openpyxl, Systemd Units, Chron

What I've Done

    Intertextual Canon Cloud (anno.wiki)

    The ICC is a web application designed to allow for collaboratively building an exhaustive and definitive repository of annotated literature. I designed, developed, deployed, and continue to maintain the project solo, managing occasionally to rope in some programmer friends for help with various features.It consists of ≈14k lines of code and ≈100k lines of code churn. The backend uses Flask/SQLAlchemy. The frontend uses Jinja2, Sass, and VanillaJS. I also had to write several ETL data pipelines for processing Project Gutenberg texts. It is deployed via Heroku. I learned the entire web application life cycle on this project, and it continues to teach me. It is currently maintained at https://anno.wiki. I am also currently working on a second iteration of the site.

    • Primarily coded in Flask/SQLAlchemy
    • Wrote many ETL data pipeline scripts for processing Project Gutenberg text files into custom JSON formats
    • Heroku for platform
    • MariaDB for database
    • Elasticsearch for search
    • Vanilla JavaScript for AJAX and progressive enhancements
    • Talisman for CSRF security
    • pytest for testing
    • WTForms for forms

    Euler Sci Website

    This was a pretty simple project but turned out so beautiful I love it. It was also one of my first paid projects and helped me acquire the contract for working on an interface for a laser system.

    • Static web site designed in GatsbyJS
    • Graphics work done in photoshop
    • Color palette designed through coolors.co

    Amazon BuyBox Notifier

    This project was for a client who sells various vitamins on Amazon. I developed, and redeveloped over several successive weeks, a rather large web scraping/api scraping system that gleans information about Amazon Marketplace Offers on products the client is selling from multiple data sources.

    Because of the increasing complexity of the project it ended up becoming rather large, with a decent amount of features:

    • multithreading capabilities
    • a token tracking system to make sure to stay within the rate limit
    • header spoofing and proxy rotation
    • near real time updates for various settings including which ASINS to track and notification endpoints using cloud file storage
    • a PostgreSQL database for tracking data (using SQLAlchemy as the ORM, of course).
    • database synchronization with an Elasticsearch instance for a Kibana dashboard

    The software was deployed to an Arch Linux VPS and used a Systemd Unit to run. I've done automated data gathering (mostly api/web scraping) before, but never on this level. I had a lot of fun, building from the ground up. I'm especially glad for having learned to do multithreading at a decent level.


    Language Map

    This project was inspired by a trend on r/languagelearning where people would post world maps with the countries colored by the languages they spoke. I decided to build an actual dedicated web app that would automatically color the countries themselves.

    To do this required a lot of data gathering and cleaning. Doing that itself was an arduous process that had a lot of false starts. The final method I settled on was to use Google Sheets importhtml function, which could be used to import Wikipedia tables to a spreadsheet. To automate this I used Python and gspread.

    It was definitely a lesson in data gathering and cleaning.

    • Extensive data gathering and web scraping mission detailed here
    • BeautifulSoup4, requests, PubProxy, scrapestack
    • gspread, Google Sheets API
    • pandas, csv, json
    • ReactJS
    • React Simple Maps
    • Material-UI
    • Hosted on Netlify

    Roastuino

    As part of my continuing obsession with coffee I discovered that I could get a lot cheaper specialty coffee, roasted closer to my personal taste, from the regions I prefer (East Africa), by just roasting myself. Sweet Maria's provides troves of information about how to home roast, and one of the suggestions they make for first timers is to use an inexpensive popcorn popper. They recommend the Nostalgia.

    So I bought it, and was dissatisfied with the lack of control I had. I found out that there are a not insignificant number of home roasters who tried hooking up a PID controller to their popcorn popper (or generally modding it), and I found this decent write up on how to use an Arduino to hook up the heating element to a 40A relay and use a thermocouple with an RTU Modbus implementation to communicate with Artisan Scope, a professional grade open source program written in Python for monitoring and controlling roast parameters (more meant for large machines, but...)

    There are some problems with Lukas' write up. Namely, that Artisan's internal PID system is really hard to get working. I ended up building in an internal PID using an Arduino library (you can see my code here). So Artisan just sends the PID values and the SV and monitors the temperature. I also found it unnecessary to use a Mosfet between the Arduino and the relay. And for debugging, I added an LCD screen (which is how I discovered that Artisan's PID was never working in the first place).

    This project was insane fun. As a programmer, I often get a kick out of some piece of software working. But making something do stuff in meatspace is a rush I'd never experienced before. It was also the first time I'd done real soldering. It was a lot easier than I expected.

    • Arduino Uno R3 from Elegoo
    • MAX31855 Thermocouple Amplifier breakout board from Adafruit with a K-type thermocouple
    • HD44780 LCD
    • Inkbird 40DA SSR
    • Artisan Scope
    • Various Arduino libraries

    Duvall Invitational Realtime Scoring Site

    My father hosts a golf tournament every year known as the DuVall Invitational. It's been running for over fifty years and is mostly played by men in the construction industry and their friends. But as with everything, using the web allowed them to improve the general quality of life in the tournament.

    They use a master Excel spreadsheet to keep track of scoring, auction data, and handicaps that is carried over from year to year. A year or two ago I made some massive improvements to the software infrastructure they use.

    First, I remade their spreadsheet from scratch, as it had become a crufty mess of tangled code, with IF(IF(IF())) style formulas that stretched over a hundred characters long with excruciatingly difficult logic. I upgraded them to using the latest version of Excel which allowed me to program it using modern Dynamic Arrays and Named Ranges. This made drop down selection a lot easier, so that the list of options in a given cell could be checked against the sheet itself to filter out used options.

    I also created a standalone handicap spreadsheet that allows them to dynamically calculate handicaps by weighting scores from the previous 5 years without having to mess with moving columns and rewriting formulas.

    Then I found the lightest-weight static site generator I could, Sergey, to generate a static site that used a little bit of JavaScript to make keeping track of the API keys for the spreadsheet stored in Microsoft OneDrive easier. The result is that my father only has to make changes to a spreadsheet that he uploads to OneDrive and those changes will dynamically update on the webpage in embedded frames that point to specific ranges on that spreadsheet. They find it incredibly helpful to check up on how their competition is doing while out on the course using their smartphones.

    I even made a PR upstream to Sergey when I found that the development server automatically exposes to the network using 0.0.0.0. Unfortunately, the maintainer still hasn't merged it. Luckily, it's lightweight enough that I can maintain the SSG myself without worrying about it being no longer maintained.

    All the code for the site can be found at malan88/duvall.

    • JavaScript
    • HTML
    • Sass
    • Sergey
    • Excel
    • Microsoft OneDrive

    Letters of John and Abigail Adams Twitterbot

    This project was inspired by a photo that made the rounds of Twitter in the wake of the protests in June 2020. The photo was of John Trumbull's famous painting of the signing of the Declaration of Independence with red dots over the faces of all the slave-owning founding fathers. Only 8 or 9 of the several dozen men in the photo could boast never having owned slaves and John Adams was smack dab in the middle with his face as apparent as ever.

    I've always been a bit of a John Adams fanboy so I decided to put my Python to work making a twitterbot that would tweet lines from the letters of John and Abigail Adams, a lovely repository of romantic correspondence.

    The bot tweets out fairly regularized sentences in threads that correspond to each letter in chronological order, and then repeats. I used tweepy for the Twitter interfacing, AWS's Lambda to run the program, and AWS's DynamoDB to keep track of where the bot is. The actual letters are hosted on a static site of json files on Netlify. That simplified some of the deployment for various reasons I can't recall now.

    • Twitter handle @john_and_abbie
    • Bot to tweet the corpus of letters between John and Abigail Adams
    • Data scraped from NARA's The Adams Papers using requests and BeautifulSoup4
    • Built using Tweepy
    • Hosted as a lambda function on AWS Lambda
    • Use AWS DynamoDB to track place in corpus
    • Use Netlify to host the letters as static text files
    • urllib3 to access the static text files
    • Boto3 to access the DynamoDB tracking

    Generated Anki decks

    Learning French from English is sometimes tricky. Especially with recognizing the difference between certain words, like au-dessous vs au-dessus (below vs above). So I decided to generate an Anki deck with audio files pulled from forvo.com's API that are distinguishable by a single phoneme (the ones I find myself distinguishing). To read my process, please see the README in the Github repository.

    The other deck, the numbers deck, consists of 2500 audio files generated through IBM Cloud's text to speech generator. It is the first 2500 numbers in French with their corresponding words and numerical representation. It is sometimes hard to understand French numbers because it uses a mixed decima/vigesimal system (seventy is sixty-and-ten, eighty is four-twenties, and ninety is four-twenties-and-ten).

    These projects are simple, but I am proud of them. I seem to have helped a lot of people learning French with them.

    The minimal pairs deck, in particular, required a lot of strategizing for dealing with the forvo.com api rate limits (500 requests per day) and the combinatorial explosion of comparing minimal pairs.

    • Generated an Anki deck for recognizing phonemes difficult to hear for an American English speaker using “minimal pairs,” words in French which differ by only one phoneme using the forvo.com api. Almost 600 downloads and 3 positive reviews.
    • Also generated an Anki deck for training the ear to hear difficult French numbers (which use a mixed vigesimal, or base-20, system) using IBM Cloud's Text to Speech renderer and the num2words Python library. Over 500 downloads and 3 positive reviews.
    • Used Requests for automated downloading of the audio files
    • Used BeautifulSoup4, requests for scraping the list of minimal pairs in an IPython session

    standingwater.io & blog.standingwater.io

    I developed this portfolio site and the accompanying blog using GatsbyJS, which I've come to increasingly love for quick static site generation. The GraphQL is, while not the most intuitive, at least pretty useful. Especially the dynamic image processing.

    The main portfolio site is built from a modified HTML5 Up template called Dimension, that was ported to Gatsby as gatsby-dimension-starter by codebushi. I have modularized it a bit, like this projects section that uses markdown files and images for each project and an array for ordering, inclusion (there are some projects I don't display for NDA reasons, unfortunately).

    HTML5 Up makes some beautiful templates, but their code is often way too tightly coupled and difficult to work with.

    The blog I built from the default starter.

    The style is inspired by my childhood home in Tampa, Florida. I initially wanted to use a background of palmettos, but it proved too visually difficult a background. I went with a swamp, instead, of which we have many. The site name is a reference to a joke name we gave to the 5 1/4 acre "estate" I grew up on. It puddled quite a lot. Almost looked like a lake.

    I also used The Noun Project to purchase the heron icon used in both sites as representative of the area. I'd prefer an Ibis, since they're lousy in the area, but a good one was hard to find.


    glendalepainting.com

    This site was built for my father's commercial painting company. It was my first project in GatsbyJS and was meant to replace glendalepainting.com. Unfortunately, my father is rather slow to update things and he's still using the original Joomla-based site.

    The gallery is perhaps the most impressive aspect. A friend of mine helped build that section.

    The main page, using react-spring for the parallax, was quite a nightmare. Parallax is not the easiest to design, but it's a dumpster fire when you have to do it for mobile devices using background images instead of sprites.

    Never again. Just don't.

    • GatsbyJS static site
    • react-spring for parallax animation
    • react-reveal for transition animations
    • Deployed using Netlify

    vote.py

    A friend of mine had never voted, nor registered to vote. So I made sure she did in 2020 by writing this little script to text her using Twilio to remind her to vote on an increasingly frequent basis as the deadline for registration drew near. She registered pretty quick, actually and it didn't run for that long. Which is lucky, because I ended up discovering a breaking bug on the month roll over.

    I ended up having to resort to good old fashioned wheedling and emotional manipulation to get her to actually vote, because she ended up just blocking the number.

    • Automated Python script to encourage and remind people to register to vote.
    • Deployed using AWS EC2 instance to enable continuous service.
    • Integrated Twilio as texting client.
    • Dynamic scheduling using datetime and sched

    gobble.py

    This was a one off quick project as a joke idea. It's a python script using the secrets module from the Standard Library to generate a password including an English word from the Electronic Freedom Foundation's Diceware list. It produces passwords that look like this: hydrant_34l;k1qjfasdf.

    The idea was to circumvent the "gobbledy-gook" social engineering vulnerability in Security Questions. Some people just put a randomly generated string for the security, but then the worry is that when an operator on a phone asks for it, they'd just accept the answer "It's just a bunch of gobbledy-gook". This script's idea is that if there's a single recognizable word in the security phrase, the operator would hopefully at least require that.

    A better solution would be to just use diceware, but where's the fun in that?


    teamup.py

    This script came about because of a family tradition of playing Christmas Trivia. The idea was to create a computerized method for achieving the closest balanced teams of arbitrary size and number.

    It takes a JSON file with team members and their skill level (this could be any numeric scale, like 1-10, 1-100, whatever), and then uses a generator to get the best match up over time. That is to say, it looks for the next best teams continuously. There's a huge diminishing return on calculating combinations, and it's extremely computationally expensive to calculate all the possible team combinations, but after about five to ten minutes you get pretty good team arrangements, better than if you just did it by hand.


Who I Am

I've always been interested in computers, developing my first website in just HTML and CSS at age 13, but was more pulled by the liberal arts. I majored in philosophy and classics at Florida State University. I only became more interested in programming as part of my ongoing obsession with literature.

As I embarked on a personal study of the tragedies of Shakespeare and the King James Bible, I became upset at the lack of definitive and exhaustive resources for deep reading in literature. Annotations on any classical text are scattered among a smattering of academic and popular books with no consistency in quality and content even within one text. They often focused on pedantic textual issues irrelevant to my interest in a deeper grasp of authorial meaning and the text's place in cultural history. So I decided to write my own application for that: a collaborative wiki for annotating classical literature: anno.wiki.

It's been almost four years since I started that project, and while the first iteration has never taken off, I've been redesigning it on and off for the past two years as a Single Page Application using a better data structure design cribbed from those who came before me in the desire to see an open source, collaborative, exhaustive, and definitive repository of literary annotations. It is, for all intents and purposes, my life's work.

As a result of that pursuit I have delved far deeper into programming and computers than I'd expected. I've studied computer security and calculus. I've studied C through K&R and Lisp through SICP. I've delved into Linux ricing and React. I've recognized the beauty in computing. In short, I began to recognize the High Culture of Open Source.

In my spare time I enjoy literature (particularly William Faulkner and Cormac McCarthy), wine, coffee, mixology, and running. And I've been known to periodically open Minecraft.

Contact

© 2021 StandingWater