We should worry, he said, when the Press is excluded from such events. The headline writer took the point further: "If the media is being gagged, start worrying".
Apart from the singular verb with a plural noun, fair enough. It is a given that a free Press is a key element of a functioning democracy.
Rifkind's article also cites Donald Trump, President Erdogan - for his persecution of Turkish journalists and for his attempts to pursue a German comedian who poked fun at him - and Jeremy Corbyn for smirking when Labour activists booed BBC political editor Laura Kuenssberg. Rifkind concludes:
This is what demagoguery is. It needn't start with jackboots and jails. It can start, as easily, with a smirk and a wink while a majority thinks "At last". Exclude the jokers, exclude the Press, exclude the experts, silence anyone who won't sing along. It is the pursuit of a clear-eyed narrative, clear of irritant grit. Watch out for it. Even if you are irritated too.
So are all those people pointing to the death of Jo Cox and urging politicians to tone down their rhetoric.
But isn't there another group that needs to look to itself? A group that has indulged in a festival of smirking and winking, silencing and excluding, not to mention mocking and insulting? The very group that sees it as its right and duty to police those in the public eye?
No sensible person wants to muzzle or gag or stifle the Press, to license it or bring it under state control. Regulator Sir Alan Moses said in his early days at the head of Ipso that we need a rumbustious, irreverent Press. We want our newspapers to be as entertaining as they are informative, to look out for us and ask awkward questions that put those in power on the spot.
What happens, though, when they peddle half-truths and downright lies? What happens when they put only one side of the story? And when they do so again and again and again?
Editors are supposed to abide by a code that demands accuracy and a clear distinction between fact, conjecture and comment.
The accuracy bit is easy in a campaign such as the one we're living through. There's always someone to express the view you want to put across and, unlike broadcasters, newspapers are not under any obligation to put the other side. Fred Bloggs says all Americans are gun-toting rednecks. He said it, we report it. It's accurate. But it's not the truth.
Indeed, when a newspaper declares that something is"the truth" or that it is "nailing lies", in most cases it is neither.
The Sun, Daily Mail, Daily Express and Daily Telegraph have never made a secret of their dislike of the EU, so their support of the Leave campaign is unsurprising.
But the way they have shown it is as worrying as Boris Johnson's exclusion of Michael Crick from that rally.
When George Osborne says leaving the EU would cost families thousands of pounds, it is part of "Project Fear". When Boris Johnson says we'll have to pay an extra £2.4bn to Brussels if we stay in, it is further evidence of the need to vote Out.
When Ian Botham backs Brexit, it's a "good old Beefy" picture story. When Professor Stephen Hawking backs Remain, it's a blob par at the foot of the page.
The "Project Fear" bandwagon started rolling early in the campaign, when the focus was squarely on the economy.
But then a much gaudier wagon came over the hill and, beguiled by the giant glittery letters IMMIGRATION over the driver's bench, our fab four couldn't wait to scramble aboard.
Turks, Romanians, Iraqis, Syrians, Afghans, Albanians: millions of them apparently want to abandon their homelands and settle in the English countryside - and only leaving the EU will stop them.
No claim was too preposterous, no figure too huge to print. And when our intrepid reporters were caught in the 120pt splash caps lie? Just put a par of 8pt on page two under "corrections and clarifications" and a bit on the bottom of the story on the website.
This all matters. These four papers have a combined readership of somewhere between 10 and 12 million, but beyond that, their front pages are seen by millions more when people go to the supermarket, to buy sweets or cigarettes in the newsagents, to pay for petrol at the service station. They are shown on the late-night news bulletins and discussed on radio and television.
The front pages thus have an influence way beyond the rest of the paper. Passers-by who saw the "We're from Europe, let us in" or "12m Turks want to come to Britain" headings won't have seen the corrections published inside much later. The stories have had their impact, the false message has reached its target.
On May 23, I decided to start monitoring every paid-for mainstream national newspaper's referendum coverage for the final month of the campaign (you can see the results by clicking on any of the front pages on the home page). Had I had half an idea how draining the exercise would prove, I would never have embarked upon it. But the imbalance in the rightwing papers and the relative ambivalence of the left-leaning Mirror and Guardian made me gasp.
In masochistic mood, I then resolved to look at everything one paper - the Mail - had written about the referendum since the date was set in February. The cynicism was even more breathtaking. You can see how the battle lines were drawn on this blog post.
The fact that I focused on the Mail does not mean it was the only - or even the worst - sinner.
The Express has failed to put the Remain case anywhere, other than to report "outrage" and "fury" over its arguments. Over the past week or so it has run a daily back-of-the-book spread labelled "Why I'm voting Leave". No counter-view has been countenanced.
The Sun was rebuked by Ipso for its "Queen supports Brexit" front page. It carried the obligatory correction, but the editor promptly said he stood by what he'd published - knocking for six any hope that the new regulator would be seen as strong enough to restrain the Press and so save it from its detractors.
The Telegraph has at least allowed columnists to express a pro-Remain opinion.
For the past four months our two best-selling newspapers have poured buckets of manure over the Prime Minister they moved heaven and earth to get re-elected. They have played fast and loose with facts (which are inevitably thin on the ground, since no one can predict what will happen next week). They have discounted the opinion of every politician, financier, economist, businessman, scientist or academic who counsels caution - and God help any "luvvie" who dares to open their mouth. "Shove it in your cakehole", as the Sun so charmingly put it.
These are the very two papers that shouted loudest when the Supreme Court ruled that they couldn't publish the name of a celebrity whose sexual antics had been touted around the street. A privacy law by the back door?
They were right.
Yes, anyone who cared knew by that stage who the fuss was all about and, yes, the whole legal challenge started out of prurience. But it was a dangerous precedent.
There is nothing wrong with (and much to admire about) a Press that cocks a snook at authority, that refuses to be cowed. But if you want to be admitted into people's homes, to persuade the reader that you are their trusty friend, you have to be responsible and trustworthy.
Goodness knows what these papers will do on Friday if there is a vote to stay in the EU - or, indeed, if there is a vote to leave and it turns out that they have helped to bring down the Government they were so keen to elect last year.
Do they have half an idea of what is likely to happen next? Or even know what they want? An administration led by Boris Johnson? Another general election that Labour - under a new leader - might win?
They rubbish Osborne and Darling's warnings about the need for an emergency budget in the event of a Leave vote.Yet (in company with pro-Remain papers) they blithely publish a Leave "manifesto" that promises Bills not only to get out of Europe but also to abolish VAT on domestic energy and other fantasy measures as though Gove and Johnson are an official opposition in a conventional general election. How will these Bills be put before Parliament?
In all their huffing and puffing about "the facts" and "the truth", papers that claim to be "on our side" and "looking out for ordinary people" have failed to ask the simplest questions or give their readers any clear picture of the sequence of events that is likely to follow the vote on Thursday.
What's the solution?
Ipso has the power to fine a recalcitrant paper £1m - but we all know it won't use it. Even if it did, papers that passionately believe that the EU is a Bad Thing - and that the end of getting out justifies almost any means - would probably consider half of Paul Dacre's annual income a price worth paying.
So retrospective punishment is unlikely to be meted out and would in any event be ineffective. It's too late. The damage has been done - to the reputation of the Press, and possibly to democracy itself.
Licensing is beyond the pale. So is any form of state regulation, however "triple locked" it may be.
Some call for Leveson 2, but that is supposed to deal with relations between the police and the Press and is hardly likely to result in a more effective form of regulation.
I really wish I had an answer.
Maybe Hugo Rifkind can offer one. What do we do when it's the "muzzled" Press that's wearing the jackboots?