Over a long time now, I have been giving my agents, mostly Claude now, more and more work, not less.

BUT I had to learn — the hard way — what discipline that requires.

Two years ago, working with earlier-generation AI agents, I would let them run too free during development.

The output looked productive. I watched the work appear and said "great, test it."

Hours and days and weeks later, I was debugging architecture decisions I never consciously made.

The agent had taken them while I was embracing the velocity.

I'd developed a system I owned on paper, but I didn't own fully enough in my own head.

What I did to get out of the holes that I made for myself:

1. Plan the architecture first. Before any code, agree on the shape together.
2. Agree on the plan specifically. Not "go ahead" — "proceed with steps 1, 2, and 3 as written."
3. Keep the plan living. Revisit it on the page when reality fights back.
4. Track status and revise explicitly. Don't drift — re-agree.

Here's what surprised me: this discipline matters MORE with Higher Reasoning AI, not less.

Weaker agents produced visibly off work. The drift was easy to catch.

The current generation of HIGH reasoning AI agents produce work that looks brilliant on first read:

* clean diffs,
* passing tests,
* good names.

That's exactly when it's easiest to sign off without staying in agreement.

And that's when you ship architecture you didn't consciously decide.

That's why I've had to develop many other development principles also ...

that my agents and I work from TOGETHER!

The proof has been in the applications that I am developing now, like PostalDataPI, my postal code API:

- Zero production rollbacks in 24+ days since our architecture cutover
- Zero errors on the production service in the last six days
- 137 tests, all green, on a single shared fixture (an architecture we agreed on together first)
- Six pieces of work shipped end-to-end in one Saturday session, no rollbacks

The output volume isn't despite the discipline. It's because of it.

I hope this helped SOMEONE out there!

Until next time,
Thom

PostalDataPI promised our customers a sub-5ms server response …

Then we started watching it slip.

Ugh!

How did we know? We measure our stuff.

The culprit was serverless cold starts.

Our proprietary fast delivery system to 246 countries & territories for a million+ postal codes needs special system features.

On serverless, every cold function spin-up meant rebuilding things that we needed from scratch.

The first request after a quiet minute could take 900 milliseconds or more instead of the single digits we were promising.

We had two choices: water down the promise, or rebuild around it.

We REBUILT.

We got CREATIVE!

Our new solution stays warm ‘’‘forever’‘’.

Today, a real ZIP lookup looks like what our demo page is showing in the post image.

The part of the work that we can control took 244 microseconds for this example — well under our 5-millisecond promise WITH significant headroom.

The 143ms round-trip is almost entirely the internet, and humans can't feel that either. However, for certain programmatic operations, that can slow things down too. But we made sure that WE ARE NOT the thing you or your customers are waiting on.

For a SaaS doing 100,000 postal lookups a month, sub-millisecond responses save real time — time their own users would otherwise spend watching a spinner. That compounds quickly.

Try it yourself at "postaldatapi dot com" and choose "Demo" in the NavBar — type a real ZIP, type a fake one (it'll warn you it's invalid), watch the microsecond-precision metrics.

The promise stands. The internet is now the limit, not us. AND we are seeking ways to reduce EVEN THAT TOO ASAP!

Until next time,
Thom

PostalDataPI is a global postal code API covering 240+ countries with:

* rich metadata,
* sub-5ms responses,
* one flat rate,
* no tiers.

I was passionate to build it the way I would want it:

* If I was a business needing it;
* If I was the developer incorporating it.

I built the whole thing from scratch.

I followed principles that I had learned from my good friend's books and personal examples - Denis Rothman (Transformers Book Author)

I ALSO HUMBLY realized that I had to adopt and create a whole new set of best practices for this new age.

I hit MANY hurdles and roadblocks, but I learned to get around them adding to my hard earned and growing set of best practices.

I recently launched.

Revenue is zero. I expected this and am now working to build it.

I'm not posting today to pretend I've "made it."

I haven't - YET!

I am STILL in the trenches — SEO, API directories, HackerNews, building country pages, recording videos, and going to the gym between deploys.

But I'm building something that is truly MINE for the first time.

It feels different. It feels better.

Along the way, family needed help — so, while I was still in the middle of developing PostalDataPI, I also took on the role of Director of Technology for Fungaia Coffee Company, because, apparently, building one thing from scratch wasn't enough of a challenge.

PostalDataPI is launching on DevHunt this week.

If PostalDataPI looks useful to you, a vote would mean the world to me. Link in the first comment.

If you're a company or developer who needs postal code data for any country visit us.

1,000 free queries, no credit card needed at signup, the best value in this space.

If you're someone who went dark to build something too — I'd love to hear your story in the comments.

I missed this community. I hope to contribute in new and better ways and be an ever better version of MORE TOGETHER.

Links:
@PostalDataPI — postaldatapi.com DevHunt (voting opens midnight MDT tonight, then 24 hours of voting) — https://devhunt.org/tool/postaldatapi YouTube 60-second demo —

YouTube full tutorial —

1,000 free queries on signup. No credit card. 240+ countries. Sub-5ms.

Why? Tons of strength training in my off time AND my work, which is super cool has crossed into a realm that is PROPRRIETARY.

Usually, I'd start a new project or portion of a project with a general post on that topic, but, lately, that's been not as possible.

However, I still love to share what I am learning, or HAVE learned if there's a chance it could help others.

In that vane, I answer many direct messages each day.

Due to the times and the current needs, I find myself sending a message like the one below More and MORE FREQUENTLY.

I wanted to share the most common version of this message in a post in HOPES that it would help many others.

Please share in the comments any additional or contrary advice you'd add.

"""

I get requests to help people get a job in one way or another!

I am honored that you'd consider asking me for my help.

I hope the plan below helps you as much as it's helped me.

I HIGHLY recommend having outstanding profiles on LinkedIn, Indeed, and possibly Glassdoor too.

Make sure your profiles are VERY good and match your very good and updated resume.

Then apply to a lot of roles thru those platforms as you find roles that you'd like.

I also highly recommend to NOT take any lack of communications from those companies personal.

Also don't take any rejections personally. We ALL get these.

When you get interviews and don't get an offer, PLEASE also do NOT take that personally.

You interview to learn how to interview better and to find out what you need to learn more about.

I hope this is helping you.

"""

Until next time,

Thom

Subscribe now

Share

WARNING: This is a "Data Science Philosophy" Post

CONFESSION: Thom LOVES the "One-Punch Man" anime / manga!

What does a manga / anime character have to do with "Data Science Philosophy"?

It has to do with career questions I get,

"Thom, Should I get an advanced degree in data science OR just take good online video courses?"

Let's look at Saitama - '''One-Punch Man'''.

In his world, Saitama is THE MOST POWERFUL HERO by a HUGE margin!

How did he become so powerful?

He trained, using very basic exercises, over and over, consistently, for 3 straight years.

When he explains this to people, they never believe him.

It sounds too basic - too simple.

How could such basic training result in a common man becoming so powerful?

For this reason -

""" Patient and consist efforts yield powerful individuals! """

Saitama didn't join the Hero Association (HA) in his world when he decided he wanted to BEcome a hero. The day that he decided that he wanted to BE a Hero, he started DOing heroic acts to save people from villains and monsters and also held consistently to his training regime. During his three years of training, no one knew all the amazing work that he'd done to reduce the number of monsters and villains in the world. In fact, many times they were accidentally attributed to another hero who has no abilities. Slowly, over time, with the help of a hero that wanted to be Saitama's disciple, Saitama learns how to make money as a hero. Saitama breaks all records in the physical portion of his HA exam to become a registered hero, but he does very poorly on the written exam. Why? He doesn't know about HA policies. He's already been training and defeating monsters for 3+ years. After that HA exam, his disciple is ranked as a top hero, and Saitama is put in the bottom class. However, slowly, he rises up the ranks and people start to learn that it is Saitama that is the STRONGEST hero. Many are slow to accept this, but Saitama's disciple is relentless in making sure that people know about all the good that he's done, because he simply continues to defeat every villian and monster.

What does this mean for us that want to BEcome great data scientists? Well, in my humble opinion, it means that NOTHING can replace the importance of our own consistent and patient self-learning on how to BEcome great data scientists on our own and nothing can replace the importance of DOing real world data science work on our own. We CAN get more visibility and make more money along the way IF we are careful to make sure the world sees our work. AND it also can't hurt if we have an advanced degree. It is one effective way to get more attention when seeking roles. However, such a degree is no actual measure of our abilities to apply data science successfully in the real world. That is attained through BEing a continual student of, and DOing continual projects in, data science.

Until next time,
Thom

Subscribe now

Share Data Science with Thom

I was taking probability and statistics. Personal computers were VERY rare. I used campus computers OR a programmable calculator.

I loved my HP Programmable calculator.

I will confess that I struggled to appreciate the more conceptual points of probability and statistics - WAIT! I still struggle with them! But at least those struggles are at deeper conceptual levels now.

If you ever hear ANYONE say that data scientists don't need to know much statistics, ... well, let me just say, IF you want to be a really good data scientist, keep fighting to understand probability and statistics MO BETTUH!

How do you start toward that goal?

I’d like to help you. First, go to my Statistics_with_Python repo on DagsHub. Find the tritium of files called named “Mean_STD_PDF_CDF_with_Python” - you can choose ipynb (notebook), html, or PDF formats.

Those files work toward integrating PDFs (Probability Distribution Functions) to calculate your own CDF.

This is what I programmed into that old and powerful HP programmable calculator, and that was one of the PIVOTAL exercises that helped me to understand probability and statistics a lot better AND to appreciate the value of code from scratch exercises.

PLEASE work through the notebook, html, or PDF yourself, and make it better for you!

I hope this helps many of you a ton!

Until next time,
Thom

As we go from machine learning on the way to deep learning, you will see how powerful PyTorch tensors and PyTorch are.

This is a first step on model training with PyTorch and its Tensors.

I can assure you that if you haven't seen this basic exercise with PyTorch on a very basic machine learning problem, you will be amazed at the elegance of model training with PyTorch.

I am excited about the time savings - not only in training speed, but also in coding models elegantly!

PyTorch brings many benefits to our data analytics and data science work.

Find the code and explanations in the “7_Modeling” section of my PyTorch_Journey repo on DagsHub. 

I hope you will clone it and follow me on this journey to use PyTorch effectively.

Until next time,
Thom

Leave a comment

Share Data Science with Thom

Subscribe now

Ambrose, my brainiac dog, is pictured below.

I hope you'll join this journey with me by cloning the repo and helping me to make the documentation of this journey better.

Remember to follow the repo and do fresh pulls on the repo after you clone it to get updates and corrections and improvements over time.

Until Next Time,
Thom

Subscribe now

Share

Share Data Science with Thom

In my ongoing PyTorch journey, I know I will want to visualize neural networks that I study.

Which tool to use?

Manim, which is used by the famous YouTube Channel 3Blue1Brown, would be great for this.

BUT, I have a ton more experience doing such things with Visual Python, and it's intuitive to this "physics and data based" modeler.

So, I put together this initial class for a starting point.

I am eager to add dynamic indicators for weight changes during training and indicators on the neurons of activation values and such.

If you want to follow the repo for this, it's on my, ThomIves, DagsHub and is called Neural_Net_Visualizer.

Clone the repo, and I hope to be making updates to it ASAP.

I've attempted to make a simple input scheme to keep this tool extremely flexible.

I hope it helps you to "ABV" in your neural network studies.

Until next time,
Thom

In my ongoing PyTorch journey, I know I will want to visualize neural networks that I study.

Which tool to use?

Manim, which is used by the famous YouTube Channel 3Blue1Brown, would be great for this.

BUT, I have a ton more experience doing such things with Visual Python, and it's intuitive to this "physics and data based" modeler.

So, I put together this initial Python class for a starting point.

I am eager to add dynamic indicators for weight changes during training and indicators on the neurons of activation values and such.

If you want to follow the repo for this, it's here → Neural_Net_Visualizer.

Clone the repo, and I hope to be making updates to it ASAP.

I've attempted to make a simple input scheme to keep this tool extremely flexible.

I hope it helps you to "ABV" in your neural network studies.

Until next time,
Thom

I've been using FastAPI for a while now.

It's helped my work and my team.

Of all the studies that I did, the work I show my FastAPI Learning And Dev repo has been the most helpful to me, and, from what I can tell, to others.

You want to find the files with the base name of “My_FastAPI” that have extensions for Markdown (.md), Iron Python Notebook File (.ipynb), or PDF (.pdf). These are all the same material - they are just in those formats.

I hope this material helps you!

It covers the 4 most common HTTP methods used in APIs.

FastAPI brings us HUGE advantages in Python based REST API development.

Why am I doing this?

My continuous integration and continuous deployment has become greatly streamlined by using REST APIs.

Also, the uvicorn server that most of us use with FastAPI restarts automatically and immediately when your code base has changed.

I hope this helps your work in #data #analytics #datascience and/or #artificialintelligence as much as it is helping me!

Until next time,

Thom

Link To DagsHub Repo For The Code

Have you ever been web scraping a lot ...

You come to a new page to scrape. 

You see the data on the web page. 

You run a `page = requests.get(page_url)` command, and you get ... 

NOTHING!

Ouch!

Sometimes, AND MORE AND MORE OFTEN, unless you actually load that page in a browser, you can't get to that data. 

What can we do? AUTOMATE!

Automate the web page operations with a Python program that uses Selenium, and then collect the data. 

Does this sound super cool? 

I assure you that it is. As you build up a set of tools and approaches, it becomes more and more fun.

"Wait Thom. This sounds like making bots with Python!" 

That is correct. You are essentially making a bot this way, and can grow into better bot making just by scraping data from pages this way.

It's great fun to automate the operations in a web browser on a website's pages and watch your code navigating through that website and operating it and collecting data.

The document contains a link to my DagsHub repo that contains all the starter code and setup instructions.

The attached PDF is just a conversion from my ReadMe.md in that repo. This PDF and an HTML version of this are also in the repo. 

I hope you will follow the repo, so that you can see updates and examples as I add them. 

I should be getting back to adding more soon. I had to study some other things first before coming back to advance this work.

Until next time,
Thom

I recently did a post on "What I would tell my younger self?"

The overall message was,

"Be Patient And Consistent As You Follow A Living Plan"

One young man asked me, "What is a living plan?" This is roughly what I answered him,

"Awesome question Arijit. We think that it was Dwight D Eisenhower that first said, "Plans are worthless, but planning is indispensable." Whether it was him, or he got it from elsewhere is not so important. But what do we do with this statement? Well, if you are planning for battle, and most of us are planning for "a type" of battle when we are planning out our efforts to reach our goals and learn and grow ourselves. We VERY often find that the first plan sucks!

Why?

Because we really hadn't entered the battle yet.

BUT THE PLANNING that we did to create the plan, which includes the information that we gathered and the things that we learned is still valuable!

That first effort with our first plan gives important information - it's feedback.

We tried that plan, and this is what happened.

I think we must ask these things:

1. Do I need to try longer or harder,

2. Do I need to change what I am doing, or

3. Should I do a little bit of both?

We make our best estimates based on the information we gathered from the first implementation of our first plan, we revise that plan, and we act on our revised plan.

This is what it means for the plan to be LIVING.

If we give up based on the results of our efforts following our original plans, we will never achieve anything.

Learn, revise, and try again, repeat.

I hope this is a help to us all.

Until next time,

Thom