You could put the cable link down low easily enough. Taller brackets coming up, weld a through-tube at the pivot point with about 1/2 inch stick-out on the side the cable attaches. At the bottom, weld a small stud off the side of the shifter tube, drill a hole at the end, washer, cable eyelet, washer, spring clip. (that's the attachment type I used at the engine end, makes it really easy to remove for engine/chain work.)
My linkage point is high because it was the only place I could get a straight shot back to the shift lever on the engine through all the frame tubing. (had to do some cutting even to get that.
Pics---remember, with the layout you're considering, forward with the lever will be your downshift, and pulling back will be your upshift.
Don't mind the dirt--only pressure washed it once after the race