That assumption is correct. Networking can help create visibility and context, but it rarely compensates for a profile that is not competitive or an interview performance that is not strong enough.
The candidates who make the jump tend to have a clear story: strong performance at Deloitte, meaningful client impact, increasing responsibility, and a reason for moving that is more thoughtful than simply wanting a more prestigious brand. Those elements make the experienced-hire application easier to understand.
Timing matters too. Experienced hiring can open and close based on office demand, so a good candidate may get very different outcomes six months apart. That is why targeted networking is useful: it helps you understand where demand actually exists.
Once you get the interview, the process largely resets. Your Deloitte background got you into the room; the case and fit interviews determine whether you convert. A useful reference is experienced-hire criteria in consulting if you want to go deeper on that part.