Taxi Lines
If you're in any popular place and haven't brought your own transportation, you will have to face the taxi line to get anywhere else except on foot.
The taxi line is the nice and equitable way the taxi drivers have worked out to share the limited fares available. The taxi drivers simply wait in line according to when they arrive and the next fare goes to the person first in line. They have the right to set the price and refuse service and nobody behind them can take the fare, although sometimes this rule will be bent.
Now, since there are far, far more taxis than make any sense on the road, there just aren't enough fares to keep them busy. This makes the taxi lines, literally, stretch for blocks in Patong while there are only smaller milling hordes in Phuket Town. Trying to get the taxi driver first in line to give up his place, which he has probably waited 2 hours to get to, for any sort of reasonable fare is impossible, so you end up paying a minimum of 300 or so Baht to even go a few blocks. What's more, it's the responsibility of everybody in the line to harass foreigners with "where you go! Taxi!" as they walk along the line.
In Phuket Town, if you really need to get a ride, the problem can be avoided by walking a few blocks from the main tourist area and flagging down a motorbike taxi or a tuk-tuk on the move. Since you're not taking them out of their precious line, you'll get a much better rate. Patong is the absolutely the worst spot and the whole area is so locked down that there's no point in trying to avoid it. You'd have to walk to Kamala to get a free agent.
The "public" transportation options are really the worst thing on the island. There has been talk for years about getting metered taxis in, but the tuk-tuks here operate essentially as a mafia, so that has slowed everything to a crawl. They've gotten so bold as to roadblock the US military on shore leave. If at all possible, I recommend either renting a bike or car. If you take more than even two or three short trips in a day, you'll have paid for it and avoided the frustration of being ripped off and hassled.