Downside 1: As the speed is reduced, the delivered torque is also reduced. At a certain point, there is not enough torque to sustain the operation.
Downside 2: As the speed is reduced, the ability to cool the motor is often also reduced. Many motors have built in fans which depend on a certain speed to deliver adequate air volume. When spinning substantially slower, there is not sufficient volume of air.
One good example of motor using a VFD is the spindle drive on your "traditional" lathe. My PowerMatic lathe has such an option. But, it can only be used down to a certain speed. Often times, this is around 75 RPM which is far too fast for a rose engine lathe.
Another example is the speed control available for routers and other fixed speed tools. Rockler sells one, item number 25278.
Stepper motors are great for spindle rotation on the rose engine lathe, and for other uses such as linear movement (e.g., a curvilinear slide).
Stepper motors can run very slowly: I have designed implementations which deliver speeds at 4 to 5 minutes per revolution. And, at this speed there is no loss of torque.
The retention of torque is a very key aspect as it allows the turner to achieve great results in their cuts on the rose engine lathe. Many report that they are far better than were ever achieved using hand-cranked operations.
Downside: The key downside to using stepper motors is that they require a bit more knowledge to setup. Controllers like the Pololu Tic 36v4 are available at a reasonable cost and are easy to hook up. Tuning them can be an endeavor, but is not terribly painful. (The instructions in this document outline the approach used for the MDF Rose Engine Lathe 2.0. ) But once it is setup, it works wonderfully!
The MDF Rose Engine Lathe 2.0 is designed to use stepper motors, and the controls are already developed for you.