Some, such as this one called Regadrella has a more vase-like shape..
For size comparison, here's the stalk versus the ROV. EASILY 2m long! (6 FEET!)
This one gives you some idea of how the stalk looks on living glass sponges. These aren't the same as the one above..but general notion..
Did you know that glass sponges are actually studied as inspiration for better fiber optics?? (here)