Yes, with somebody! You can stay very small and not require much but you'll probably need a local business license. You may need to file a fictitious name statement. You may need a resale permit if you are selling a product or service in some states. And, if you make $1, you'll have to let the IRS know about it!
Banks need to know you're legit in order to open an account.
But don't go overboard until you're ready. I see little service companies (dog walking, for example) spend $10,000 getting LLCs and licenses and tons of "legal" stuff. Be sensible.
Not necessarily, a business can take off from an idea and it is about the business catching up the fire in the market. So if you start doing well you should register your company immediately before someone else takes the name away from you.
But before you start, make sure that the name for your business is available to register. If you do not register, you should register as your brand is in the market and catching the fire.
Absolutely, if you have a long-term plan and you eventually have a great finance to support your business for 2 years. You should get the REGISTRATION done because you might not take the risk of losing the name and investing in the brand image.