I've no experience with fishing, but I guess you won't be wanting to hump your fishing gear too far from the car? So, coastal car access is gonna be Porlock, Lynmouth, maybe Hunters Inn (1 Mike walk), or Combe Martin. Further west than that and your getting away from Exmoor.
They coast path west of Porlock is nicely wooded and it's easy to get up on the moor.
Lynmouth towards Woody Bay is good too, but to get to the moor you'll have to negotiate The lynn valley, which is rather nice in itself.
PM me if you want more specifics.