Sir Gavin said he asked the constituent he was speaking to stay on the line as a “comfort” until Mr Parry stopped following him.
The court heard the incidents left him “fretful” and “fearful” about his safety.
“When someone is acting in such an extreme way… you don’t really know what they’re capable of doing,” he said.
District judge Tan Ikram said there was no case to answer for the charge of impersonating a police officer.
Olivia Beach, defending, argued the evidence was “so poor or weak” that it would be unsafe to convict Parry on the charge.
The judge upheld her submission.
The defendant appeared at court with Piers Corbyn, brother to former Labour leader Jeremy Corbyn.
Sir Gavin served as defence secretary before becoming education secretary under the then-PM Boris Johnson in 2019.
The trial continues.
