Yes, you can go for a hike inside the city of San Francisco [1]. Most of the parks in the City offer hiking trails to suit various tastes and ability levels. The City also boasts some longer and more interesting trails that present serious hikers with a real challenge.
Links:
[1] http://www.moon.com/destinations/california/san-francisco-and-the-bay-area