Sounds like you need to walk back tune . 20 yard pin at 20,30,40 yrds .
If pattern goes to left move restbto the right slightly. If pattern moves to right move rest slightly left .
Then check your 20 yrd pin after moving rest . And then try 50 yrds again .