I'm using Phalcon 2.0.8 with php 5
How to upgrade it to 3.0.1?
should I update php 5 to 7 first? And if I update Phalcon 2.0.8 to 3.0.1, will my phalcon 2.0.8 applications still work?
Check blog post about changes in 3.0.0 - https://blog.phalcon.io/post/phalcon-3-0-0-released there are written deprecated things too.
To update it just install it as usual from repository or using apt or yum.
If you will update your code - then it will work.
Well if you want to use php 7 - then yes, update to php 7 first.
Test your app first. This is a major upgrade. I would vote for PHP 5.6 and latest Phalcon 3.0.x. I use that combo and it works perfectly.