I'm pretty sure it was British, maybe 60's. I remember it in black and white.

A man hires a detective to follow his wife around. She's not unfaithful, just bored. The wife and the detective form an unusual friendship, based on him following her around.

specific scenes I remember:
The phone rings, wife picks it up, there is silence on the line. She smiles and says, "Is that you?"
Husband confronts his wife about "the other", and describes him as a handsome man with the air of a diplomat. She replies, "No, he's not. He's a frumpy man in a wrinkled raincoat, eating macaroons out of a paper bag."

Can anyone identify it? I've been going nuts trying to Google it with no success.

It sounds a lot like The Public Eye (1972) with Mia Farrow, although that wasn't a B&W film.

Quite often, while watching a movie from the 60s or 70s with my dad (most recently The Out-of-Towners), he'll remark that he remembers the movie being in black-and-white. Which it probably was, because most likely he'd been watching it on the late show on a black-and-white TV.

That's it! Thanks.