EVE New Citizens Q&A

You get more than 5 probes by training Astrometrics higher. Get Astro 4, that unlocks Rangefinding which really improves results.

For the OP though:
I see your problem quite clearly, in the second picture. You are using too low an AU setting.
When you get a red sphere, it means that only one probe is getting an adequate signal from the target. It also means that the target is located somewhere on the surface of that sphere (roughly). In that picture, the area of overlap within the center of your scan spheres is entirely inside the red sphere. That overlap is where you need your target to be, and it isn't.

At that point what I would do is use alt-drag to make the formation larger. Open it up until each probe is at least close to the surface of the red sphere, if not actually outside it. Then kick up your AU a notch, so that every probe is still inside the sphere of every other probe, *and* the red sphere is well within the overlap of the blue spheres. Scan again. This should get you a low% single target. Shift-drag to recenter your formation, alt-drag to bring the probes a bit closer to your target... but do NOT lower the AU. Scan again. This should give you a noticeably higher % than the last scan. Now it should be safe to recenter, shrink down, and drop one AU before scanning again.

The key point is you always want your target to be well within the area of overlap of all your probes. Don't start shrinking down until you have a single dot that's over 25% at least.

Oh, and the reason having the center probe at a lower AU is because of the overlap. What that does is fills the overlap area (but not much farther) with a higher-strength signal.