Software engineers often carry a strong disdain for magic.
Ironically enough, this is because they’re in one of the few professions where magic, for most intents & purposes, actually works.
It’s not uncommon for a junior programmer to be critiqued on his use of “magic strings” (string ~= non-code text) or to hear an engineer grumbling about how much “magic” some developer tool or software library does under the hood, where it can’t be easily inspected, debugged, or understood.
I’ve long believed that programmers should ~always try to understand how the libraries & tools they use work.
Here’s why I don’t think that anymore:
I recently encountered a business problem that would’ve taken me as much as years to complete if I had to understand the full implementation myself.
The task? Compare hundreds of thousands of unstructured text snippets against each other in order to determine which ones are most similar for recommendation purposes.
I could’ve spent many months learning high-level math up to advanced calculus starting from high school algebra (which I originally failed!), then studied deep learning, neural nets, transformers, & how to design a language model, then implemented a solution.
Instead I copy-pasted some example code from OpenAI that used their ada text embeddings API, tested it a bit, and once it worked, I was done.
10 hours of work, instead of 10,000.
Old me would’ve been disgusted that I was throwing my problem into a black box to be solved. Now? I’m at peace with not knowing.
There are no points awarded for difficulty.
There is only so much time to live.
Pick your battles wisely.
Through this blog, I’ll be sharing future learnings, triumphs, and perhaps even opportunities related to my work in technology & investing. I’d love to have you on my mailing list. Sign up below.
