Making Apples Financial Trend Analysis & Forecasting


Six Rules of Personal Finance, Part 1 – Best Way to Make Money

This website, and blog, focuses primarily on the analysis and interpretation of numbers.  And let’s face it, usually the numbers of greatest interest to us happen to have dollar signs (or any other currency) in front of them. For all of us, to greater or lesser degree, and to very varied times in our lives, the numbers pertaining to our own financial well-being and position are of particular interest.

I’m going to turn my attention for a while to matters of personal finance, drawn from a lifetime of experience (I’m sixty-two), rather than from formal education. I have no certificate of professional authority, but have done alright if I may say so. The ideas to follow over the next number of sessions will share lessons learned and observed; they are heartfelt. And they will reflect on how I have pursued my own (and my family’s) financial well-being. I believe strongly that as with any life lessons learned, of any variety, some will resonate much more than others simply because of our experiences, the values we cherish, and our own very personal insights, intuitions, and frameworks. Whether we’re aware of it or not, we each have our own set of rules of personal finance that we follow and that we evolve, for better or worse. I humbly present my own set of “rules” of personal finance.


  1. Best way to make money: don’t spend it

I’m going to resist the temptation here, at least for now, to wax philosophical on the purpose of money. Suffice to say that for most there’s usually a desire for more of it. And invariably, the more one makes, the more one spends. Our personal desire for things from clothing and jewelry, to houses and cars, requires some given level of income – all too often a level higher than the one we’re presently at.  In collecting the things of life, we also collect debt.

It is remarkable how much pressure we can take off ourselves by being content with less, by simplifying our lives. We will always have basic human needs to meet – food, shelter, clothing, good health. For some, that last category can be extremely expensive, particularly if you’re unlucky enough to contract a disease or malady, and live in the United States. This country has found a way to make healthcare punishingly expensive. I’m not going to suggest here that “simplifying your life” is necessarily achievable if you have major health conditions to address and struggle mightily to have the funds to treat them properly. Some have to make horrendous decisions on whether to buy certain medicines or to buy food for the table, unable to afford both.

Nonetheless, there is much to be said for asking ourselves from time to time if we could get by with less; in fact, might it indeed be possible to thrive and “prosper” on less?  One less car, nice secondhand coats, a smaller home: downsizing can change our orientation on life to free us to think more and enjoy more of the things that really matter in life – family and friends, beauty and serenity.

One of the easy mistakes we make about the things we purchase is to think that the price tag reflects the “true” cost of that item. Say you’ve got your heart set on a car that cost $10,000. Assume further that you don’t have any extra money in the bank, and you want to pay cash, and you decide to take on a second job to pay for it. You’re going to have to go out and earn $10,000 to afford it, right? Wrong! The “true” cost of that car is much higher.  Of course, there’s sales tax which perhaps adds $500. But much more significantly is that the income you earn to pay for it is in turn subject to income tax. Depending on your tax bracket, you may in fact have to earn $12,000, $13,000, or more, in order to be able to pay out a few thousand for the state and federal income tax before you have enough left over to pay for the car and its sales tax.  Of course, if you buy it on credit, there will be the additional hundreds or thousands of dollars in interest owed the bank over the years taken to pay off the debt (and of course, you’ll be paying income tax on that money earned to pay that extra interest). All this additional money that must be earned to pay for stuff is strong support for the notion that the best way to make money is to not spend it.

Years ago I read a book called The Millionaire Next Door. My memory being what it is, I only recall one central item from that 1996 bestseller. The authors’ observations then likely still hold true today: most millionaires don’t live in big houses in the expensive part of town, they live “next-door”. All too often, the folks who live in the million-dollar homes and drive the expensive cars are deeply in debt and have their own substantial financial struggles despite enjoying incomes far higher than the rest of us. Meanwhile, the millionaires next door live much simpler lives, quite contentedly (hopefully) getting by on much less, and carefully spiriting away and investing well a little bit of each paycheck. Sometimes we read in the paper of some “pauper” who bequeaths millions to a university or library or cause. Much more common though, is the family that has simply been careful with their money, ever striving to put some away all life long, to be able one day to retire at an age more to their choosing.

Making Apples of Student Test Scores

Last time we examined how school districts & communities are penalized for having a well-diversified student body.  The penalty occurs as students of color generally score much lower on standardized tests.  And those tests are often a key ingredient in measuring the “quality” of a given district, or school.

We noted the rather extreme example of a comparison of two local high schools, one here in Madison Wisconsin and one in the neighboring town of Middleton.  ACT scores for the Madison school were higher (than Middleton) for whites, higher for blacks, higher for Hispanics, higher for Asians, higher for all other races.  Yet Middleton’s overall score was higher.  How could that be?  Because Middleton has a higher percentage of students that are white, and because whites score higher, their overall average is higher than Madison.

Think about that.  You’re moving to the Madison area; your choice of location leans heavily on test scores (perhaps not the best measure, but it is nice to have an easy number to use for comparison); you look at these two schools… and you opt for Middleton as it has a higher score.  But if your child is white, wouldn’t you want to go to the Madison school since it scores higher?  If he or she were black, wouldn’t you opt for Madison?  Or if they were Hispanic, or Asian, or…?  Our world, and certainly here in America, is becoming increasingly diverse.  By choosing Madison, your child gets greater exposure to that diversity and will hopefully be better prepared for the working world, and social and cultural world, as a result.  But that overall test score may easily steer you away.  How can we fix it to reflect more accurately how these two schools compare?  Quite easily.

To demonstrate how to make the appropriate comparison of results, I will refer now to recent (2015) NAEP results for 8th grade math.  The NAEP (National Assessment of Educational Progress) is a nationwide exam that represents the best available comparative tool for measuring student performance across the U.S.  On the left hand side of the table below, the 8th grade math scores are shown.  For each racial type, Texas scores higher than Wisconsin.  Yet, Wisconsin’s overall score is higher.  When we look at the racial distribution at the right, we see the primary driver: Wisconsin has a much higher percentage of white students than Texas.  And as whites generally score higher, the overall score for Wisconsin is higher.

8th Grade Math NAEP Scores
Wisconsin Texas WI Dist TX Dist U.S. Dist
White 296 298 76% 31% 52%
Black 249 267 9% 11% 15%
Hispanic 271 277 10% 52% 24%
Asian 295 312 3% 4% 6%
All Other 283 290 2% 2% 3%
Original Overall 289 284 100% 100% 100%

The simple fix for the distortion is to treat both states as though they each have the same racial distribution.  The most appropriate distribution to apply is that for the U.S. overall.  Essentially, the weighted average score for each and every state is calculated using each state’s actual scores by race, and applying the U.S. average race distribution as weights.  Thus, the Wisconsin white score of 296 has a weight applied to it of only 52%, not 76%.  Meanwhile, the Texas 298 score for whites will also be multiplied by 52%, instead of by just 31%.

By applying the nationwide racial distribution to both states, the overall results change quite dramatically.  The Wisconsin overall score drops to 283, while Texas jumps to 288.  This result seems very reasonable, given how well each race in Texas performs compared to Wisconsin.  Not surprisingly, the state nationwide rankings change accordingly.  Wisconsin’s original ranking of 6th falls to 19th, while Texas soars from 22nd to 3rd.



Adjusted Overall 283



One final point to address: sometimes a state’s count for a given race may be so small that they are unable to report a statistically valid average.  For example, in Wisconsin that was the case for the category “2 or more races”, which represented 1.3% of the 8th grade math test takers.  When only one category is missing, one can apply algebra to “back into” the approximate score.  But if more than one category is missing, or if results are already rounded, a different approach may be more appropriate.  Although inevitably inaccurate to some slight degree, you may do just as well to take the ratio of the state’s overall score to the U.S. overall score, and apply it to the US average score for the given category.  Thus, for Wisconsin’s “2 or more races”:

Estimate  =    (WI Overall Score / U.S. Overall score) x U.S. Overall score for category

=    (289 / 282)  x  285

=    292

The Hughes Diversity Index

Ed Hughes was a recent school board member, and president, here in Madison.  He is a lawyer, and a terrific champion of public schools.  In a recent blog, Hughes describes what he calls the “Diversity Dividend”, the benefits derived by students attending schools with high racial diversity: academic, social, and civic, as well as providing better preparation for work life.

Regrettably, diversity often ends up penalizing school districts.  Parents evaluating the potential of a given school or community examine test scores as a convenient and easy measure of academic strength.  But almost invariably, students of color score lower, on average, than white students.  Consequently, the published overall average score for a given district is usually lowered as the student body becomes more diverse.

This tendency can be quite diabolical.  As example, Hughes points to the ACT scores for two local high schools, one in Madison and one in the adjacent town of Middleton.  Madison’s score for every racial group was higher than Middleton.  But because of the racial composition, the overall average for Middleton was higher; (Middleton had a higher percentage of whites, so even though they scored lower than Madison’s whites, the larger percentage helped raise their overall score above Madison).  There’s a way to report the numbers that addresses this incongruity; we’ll look at that next time.  For now, suffice it to say, you have to dig a little deeper than just one simple reported figure if you want to get a more accurate read on what the numbers have to say.

Being a blog that focuses on numbers, I wanted to describe the technique Hughes employs to measure the level of diversity.  As a starting point, he references what is called the Herfindahl-Hirschman Index, or “HHI”.  It’s an approach used in antitrust law to measure the concentration of products or services, i.e., to measure the relative lack of competitiveness.  It’s calculated by taking the market share of each participant, squaring the percentages, and adding them up.  So if a product market has four companies with shares of 50 (percent), 30, 15, and 5, the HHI would be: (50 x 50) + (30 x 30) + (15 x 15) + (5 x 5) = 2,500 + 900 + 225 + 25 = 3,650.  Another market where four firms each have a 25 share would have an HHI of 2,500 (= 625 + 625 + 625 + 625).  In its extreme form, a market with only one firm possessing a 100 share would have an HHI of 10,000 (= 100 x 100).  Clearly, the lower the score, the better: one wants more competitiveness, not less.  Similarly, to obtain the “diversity dividend”, we want to see higher diversity, not lower.  But how to measure it?

Hughes takes the HHI formula, turns it on its head, and comes up with a simple and brilliant technique for measuring diversity.  First, Hughes takes the highest possible score of 10,000, and subtracts from it the sum of the square of the racial share percentages.  Here in Wisconsin, racial mix is usually reported across five categories:  Asian, Black, Hispanic, White, & all other.  (Actually, racial mix here also includes American Indian, Pacific Isle, and Two or More Races; I’ve combined these typically very small categories into “all other”, to ensure share totals add up to 100.)  For Wisconsin, with a racial mix of 4, 9, 11, 72, & 4, that equates to:  10,000 – (16+81+121+5,184+16) = 10,000 – 5,518 = 4,482.

Next, he scales up the score so that perfect diversity adds up to 10,000.  With five categories, a perfect distribution would have all five with a 20 share, which when subtracted from 10,000 leaves 8,000.  Scaling up that score requires multiplying it by 1.25.  Stated more broadly, the adjustment entails multiplying by n/(n-1), where n is the number of categories you have.  (If you have five categories, the factor is 5/4 or 1.25; 3 categories would be 3/2 or 1.5; etc.)  The scaled up Wisconsin score is: 4,482 x 1.25 = 5,602.  To make the score less cumbersome, Hughes’ third and final step is to divide the result by 100.  So the Wisconsin score ends up simply as 56.  (Actually, the racial mixes here have been rounded; not rounded, the final score for Wisconsin comes out at 57.)

And that’s it.  I think what I’ll call here the “Hughes Diversity Index” is brilliant.  It’s easy to calculate, and so easy to comprehend and use for comparative purposes.  It can obviously be applied to any setting where you want to capture the relative mix, be it public schools, universities, corporations, communities, countries, whatever.  By the way, for the US population overall, the index is 69, while for the US student population – using averages from 4th grade and 8th grade NAEP test takers – the score is about 81.  The higher student score reflects expectations of ever-growing diversity in this country.  The Madison school district’s score was 90, the highest of all of Wisconsin’s 424 school districts.  How does your community/school district / company fare?

The Retention Rate Formula (Part 3)

Over the last two weeks, we’ve been looking at the formula used to calculate retention.  Unless an organization is able to separately track new and existing customers, the problem with the retention rate formula becomes one of determining how to treat new customers.  We found that the most commonly used formula effectively treats new customers as though they all arrive on the last day of the year.

YearEnd Formula:            Retention Rate (RR)  =  (End – New) / Start

Another formula some firms use effectively treats new customers as though they all arrive at the very beginning of the year.

NewYear Formula:          RR  =  End / (Start + New)

So what does a more accurate and “reasonable” formula look like?  Well, if you think about it, for most businesses, new customers will be coming in throughout the year.  Some will sign up on the equivalent of January 1, some will arrive December 31, and all the others will arrive all the various days in between.  On average, they’ll arrive around midyear.  So, to measure retention properly, we want a formula that treats new customers as though they arrive midyear.

A “MidYear” formula is quite easy to construct, if we just re-state our traditional formulas a little differently.  The “NewYear” formula can be revised to show that there are zero new customers being subtracted from the ending customers in the numerator, while the denominator re-states the new customers as “1 New”, meaning that it picks up all 100% of the new customers.

NewYear Formula Restated:      RR  =  (End – 0 New) / (Start + 1 New)

Similarly, the “YearEnd” formula can be restated to show all new customers subtracted from the ending in the numerator, while there are zero new customers added to the denominator.

YearEnd Formula Restated:        RR  =  (End – 1 New) / (Start + 0 New)

Well, a “midyear formula” should be the average of these two formulas, right?  In the numerator, we’ll subtract the average of 0 and 1, or ½, of the new customers.  And in the denominator, we’ll add the average of 0 and 1, or ½, of the new customers to the beginning customers.

MidYear Formula:             RR  =  (End – ½ New) / (Start + ½  New)

And voila, by treating new customers as arriving midyear, on average, we have a retention rate formula that remarkably well reflects the “true” retention rate.  Our example was for all of 2017, but it can be applied anytime during the year.  Though new customers will seldom come in on average at exactly midyear, they will probably usually come in very close to that.

We can now try out our new formula on the simple example we used previously.

MidYear Formula:            RR  =  (End – ½ New) / (Start + ½  New)

Example:                             RR = (100 – ½ x 20) / (100 + ½ x 20)

RR = (100 – 10) / (100 + 10)

RR = 90 / 110 = 81.8%

We subtract half the new customers, or 10, from the numerator, and add half of the new customers, or 10, to the denominator.  We arrive at a retention rate of 81.8%.  As one might expect, the result is roughly halfway between the 80% we got using the “YearEnd” formula, and the 83.3% obtained using the “NewYear” formula.  It’s a little more involved, but if you want to use the “right” formula for calculating the retention rate where the customer counts include new customers, the MidYear formula is the right one to use.

The Retention Rate Formula (Part 2)

Last week, we looked at the typical formula used for measuring the retention rate.  That formula is easy to use.  But it’s not that accurate.  The most commonly used retention rate formula is:

Formula #1:        Retention Rate (RR)  =  (End – New) / Start

Some firms use a different formula.  Instead of subtracting the new customers from the ending account total, they include new customers with the starting base.

Formula #2:        RR  =  End / (Start + New)

Again, we’ll demonstrate the formula under the same scenario previously used: we start and end the year with 100 customers, and add 20 new customers during the intervening 12 months.

Example:             RR = 100 / (100 + 20)    =    100 / 120    =    83.3%

You end with 100 customers; you started with 100 customers, and added 20 along the way.  Of the 120 total customers, you’ve “retained” 100 of them, for a retention rate of 100/120, or 83.3% – using Formula #2.  One of the attributes of this approach, that management would appreciate, is that you end up with a higher figure.  This formula, like #1, is also easy to calculate.  But which of these, if either, is more appropriate?

The central challenge with any retention rate formula is how to treat new customers, or more precisely, how to treat when they are new.  Let’s walk through a couple of examples, wearing an economist’s hat.

To help clarify a given problem, economists love to make assumptions.  So let’s start by assuming that it’s the end of 2017, and we’re looking back and measuring the retention rate for 2017.  Let’s further assume that all the new customers came in on the 2nd day of the year, January 2.

If all the new customers came in on the 2nd day of the year, then essentially, by the end of the year they have effectively had an entire year in which to leave, while the existing customers had the full year to leave.  In this instance, it is quite reasonable and appropriate to treat the new customers as though they were existing; after all, they have had virtually the same amount of time in which to leave as the existing customers did.  So the formula for measuring retention for 2017, under this scenario, is to take our ending customers and divide them by the starting plus new.  Let’s call this the “NewYear” formula, since all the new customers arrive at the start of the new year.

NewYear Formula:          RR  =  End / (Start + New)

Next, let’s run a different scenario where we assume that all the new customers arrive on the 2nd to last day of the year, December 30.  Are we going to want to include them in our measurement of retention?  Absolutely not; they haven’t had any time to leave yet.  So here we’ll want to subtract our new customers from the count of ending customers, while our baseline will be our starting customers.  Let’s call this the “YearEnd” formula, since all the new customers arrive at year end.

YearEnd Formula:            RR  =  (End – New) / Start

Of course, our “YearEnd” and “NewYear” formulas are identical to the “Formula #1” and “Formula #2” that are commonly used.   The primary formula, “Formula #1”, is identical to the YearEnd formula; it treats all new customers as though they came in at the very end of the year.  Our second formula treats all new customers as though they come in at the start of the year.  Clearly, neither of these formulas rest on a reasonable assumption.

Next week we’ll determine what the retention rate formula “should” be.

The Retention Rate Formula (Part 1)

The retention rate is one of the most important metrics for any organization that has customers paying on some kind of regular or subscription basis.  From wireless telecom service to insurance coverage to banking products and more, the ability to retain customers over time is critical for success.  The typical measure of retention is one that calculates the percentage of existing customers that remain over a one year period.  You start the year with 100 customers; one year later, 80 of them are still on the books: the retention rate is 80%.

That’s simple enough.  The tricky part with any retention rate formula however, is how to deal with new customers who come in during the year.  Ideally you separate them out, and indeed some companies do separately track retention on existing customers versus new.  But there are some definite challenges with that approach: the minutiae of tracking all the different types and timings of attrition can often lead to numbers that “don’t add up”; there is the complexity of providing two different measures of retention, leading to the inevitable request for a third measure that combines existing & new customers; and finally, there is the question of whether the time and cost are worth the effort.

Be that as it may, let’s presume your measure of retention is for new & existing customers combined; what formula do you use to measure retention?  Recently, I did a Google search on measuring retention, and by far the most common formula I came across was the following:

Formula #1:        Retention Rate (RR) = (End – New) / Start

Very simply, one subtracts new customers from the count of ending customers, and divides that amount by the starting customer count.  In the equation, the starting point is 1 year prior to the end point, and new customers are the count of sales during those intervening 12 months.

To demonstrate this retention formula, as well as two other formulas to follow, let’s use a simple example where we start and end the year with 100 customers, and during the year bring in 20 new customers.

Formula #1:        RR = (End – New) / Start

Example:             RR = (100 – 20) / 100

RR = 80 /100 = 80%

In this example, we start the year with 100 customers, see 20 new customers come in during the year, and end with 100 customers.  If we subtract out the new customers, we see that we’ve retained 80 customers, giving us a retention rate of 80 over 100, or 80%.  Nice.  Simple.  But unfortunately, not very accurate.  Next week, we’ll see why.