I did that counting off the days from the calendar too, once upon a time, and reasoned that if we started with Ash Wednesday as Day 1 we'd count Palm Sunday as Day 40, so Holy Week would not actually be part of Lent....

If we omit the Sundays from the count, the 40th day after Ash Wednesday (including Ash Wednesday as Day 1) would be Holy Saturday, and that works out too.