??drain off the blocking solution before adding the primary and not rinsing
with buffer in between??

I believe the answer is two-fold.
The blocking solution is used to "bind up" any nonspecific proteins that may
increase your background. You would want to maintain this attachment not
risk washing it away if the random binding was not strong. The nonspecific
binding of the serum will not interfere with the specific binding of your
antibody.
Your primary antibody should be made in the blocking solution in order to
help decrease the nonspecific background so at the blocking step you have a
tissue that has everything on it minus the antibody. When you add the
primary you are essentially not "changing" conditions. If you were to rinse
with a buffer you would change the ideal conditions for the antibody.
anita