With the house behind you, go to the shoreline and try to cast to the middle of the "pond" (i.e. as far as you can). Before you cast, where is your bait? Is it in your hand or is it hanging from the rod tip? If it is in your hand you need to change the casting style before casting (on the Steam version you press the (function) key F11 to do this.
Also, how big is your keepnet? The mad-eye buffalo gets to about 6 kg (13 lb), if the max fish size of your keepnet is less than this then you'd have to let it go (missing out on around $1000 credits), i.e. maybe waiting until you can keep a fish that big.