Plain text life, including gopher

With a bit of python, lynx, and tidy I was able to pull very clean plain text versions of my WordPress posts. The sparse HTML can be found at http://tokyogringo.myjp.net and the markdown text version can be found on my gopher site at gopher://sdf.org:70/0/users/tokyogringo/

How did I do it? This site has full text RSS for everyone’s enjoyment. No one has to actually visit https://www.prjorgensen.com in order to consume the high value content I generate. The feed contains everything needed for this plain text life. How to make use of it?

I fumbled through my first in a long time python script relying heavily on the very powerful feedparser module.

This Just In: python’s documentation is terse almost to the point of incomprehension While accurate, the documentation does not help beginning (and maybe middling) python coders get to solving problems. Oddly, the Reddits and StackExchange sites are also of limited utility as the answers there often point back to or copy the documentation.

Anyway, taking a very Unix approach I decided not to do everything in python. I know tidy for making valid HTML. I know lynx for terminal-based web browsing, and the ‘-dump’ option produces markdown versions of web pages.

Once I got the script to the point of providing the website data in a reliable and eventually parse-able way, then I turned to getting all my posts.

I cranked the RSS feed of prjorgensen.com up to 20,000 to make sure the feed briefly included all of my posts. I moved my parsing script to my MacBook Pro because I didn’t want to choke the sdf.org servers with my madness. I installed modules and localized the script to run on the MBP.

I ran the script. I checked my email. I then got up to … hmmm. The script finished in under two minutes. Suddenly I had all of my posts back to 2011 in both very clean HTML and in plain text. I synced them to their proper home. I reset my website feed back to a more reasonable number.

There are any number of improvements I can make:

  • My script does not grab images
  • I capture categories and tags from WordPress but don’t do anything useful with them
  • I need to include modifying my gophermap and my index.html (as appropriate)
  • A full text RSS feed of the plain HTML site
  • A full text RSS feed of the gopher site
  • Maybe use a static web site generator like Jekyll for the plain HTML site
  • Maybe use this for tokyogringo.com and PVCSec.com? If so, then I need to handle …
  • Media enclosures

Watch this space for the link to my script on GitHub. Which is here!

I’m pretty sure I’m going to buy a TV if only to watch Japanese ads. They. Are. Magical.

Private Transport Monopolies Will Be Bad for Everybody

Private Transport Monopolies Will Be Bad for Everybody:

Last week the transportation rumor mill pumped out a story that ride-hailing company Lyft is acquiring Motivate, the bikeshare operator behind New York’s Citi Bike, San Francisco’s GoBike, and Chicago’s Divvy Bike. The deal, which was first reported late last week by The Information, is said to be in the range of $250 million.

I’m not sounding the alarm over a $250-million acquisition, but it is worth examining how consolidation in the private transportation sector will affect the public. After all, monopolies in agriculture and healthcare have led to higher pricing, artificial demand, and antitrust strategies like price-fixing.

(Via Motherboard)

More important, isn’t this a variation of the plot of Who Framed Roger Rabbit?

Censorship in the Age of Large Cloud Providers

Censorship in the Age of Large Cloud Providers:

Whatever its current frustrations, Russia might well win in the long term. By demonstrating its willingness to suffer the temporary collateral damage of blocking major cloud providers, it prompted cloud providers to block another and more effective anti-censorship tactic, or at least accelerated the process. In April, Google and Amazon banned—and technically blocked—the practice of “domain fronting,” a trick anti-censorship tools use to get around Internet censors by pretending to be other kinds of traffic. Developers would use popular websites as a proxy, routing traffic to their own servers through another website—in this case Google.com—to fool censors into believing the traffic was intended for Google.com. The anonymous web-browsing tool Tor has used domain fronting since 2014. Signal, since 2016. Eliminating the capability is a boon to censors worldwide.

Tech giants have gotten embroiled in censorship battles for years. Sometimes they fight and sometimes they fold, but until now there have always been options. What this particular fight highlights is that internet freedom is increasingly in the hands of the world’s largest internet companies. And while freedom may have its advocates—the American Civil Liberties Union has tweeted its support for those companies, and some 12,000 people in Moscow protested against the Telegram ban—actions such as disallowing domain fronting illustrate that getting the big tech companies to sacrifice their near-term commercial interests will be an uphill battle. Apple has already removed anti-censorship apps from its Chinese app store.

In 1993, John Gilmore famously said that “The Internet interprets censorship as damage and routes around it.” That was technically true when he said it but only because the routing structure of the Internet was so distributed. As centralization increases, the Internet loses that robustness, and censorship by governments and companies becomes easier.

(Via Lawfare – Hard National Security Choices)

Congress Should Oversee America’s Wars, Not Just Authorize Them

Congress Should Oversee America’s Wars, Not Just Authorize Them:

Nearly 17 years after the 9/11 attacks, a bipartisan coalition of senators has put forward legislation that promises to overhaul the legal framework for America’s worldwide campaign against terrorism. Proponents of this measure argue the existing authorization for military force—an AUMF in wonk-speak—passed back in September 2001 has become woefully outdated. The failure to modernize it, supporters say, represents a dereliction of duty by Congress.

They have a point. The text of the 2001 AUMF no longer bears much resemblance to the wars we are fighting and that we will continue to fight for the foreseeable future. As a matter of both constitutional good practice and common sense, the case for an updated statute is clear.

The problem is that, while a new authorization is legally desirable, its real-world impact is likely to be minimal—doing little more than sanctioning military operations the executive branch is already prosecuting. Lawmakers who portray passage of an AUMF as the ultimate fulfillment of their war-powers responsibilities therefore risk elevating constitutional form over national security substance—while neglecting the far more powerful but less formal tools Congress possesses to influence America’s post-9/11 wars for the better.

That is unfortunate because the need for thoughtful, energetic congressional activism has never been greater. From Afghanistan to Syria to the Sahel, multiple complex U.S. military operations are unfolding . Members of Congress are uniquely positioned to scrutinize these efforts and the strategy underlying them, identify any flaws and failures in policy, and inject innovative or disruptive new ideas into the public debate that will make success more likely.

In the mid-2000s, for instance, it was Members of Congress from both parties who were pivotal in challenging—and eventually overhauling—the Bush administration’s strategy in Iraq.

During those years, senators like John McCain and Joseph Lieberman (for whom we worked at the time) regularly traveled to the Middle East, meeting with military commanders and frontline forces, while back in Washington, they engaged not only the administration officials responsible for Iraq, but also think tank scholars, reporters, and visiting foreign leaders.

These interactions both convinced them that White House claims about the war’s progress were mistaken and brought them into contact with dissidents, inside and outside government, who were arguing for an alternative strategy—a population-centric counterinsurgency campaign, backed by more forces. McCain, Lieberman and a few others then used the congressional bully pulpit to advocate for this approach, which the Bush Administration eventually embraced. The result was the 2007 “surge” that stabilized Iraq.

Of course not all ideas emanating from Capitol Hill are good ones, to put it mildly. Nor is it the appropriate role of Congress to micromanage the day-to-day conduct of a war through binding legislation.

Rather, the mission of Congress should be to provide smart, determined oversight—asking tough, well-informed questions, illuminating and demanding accountability for failures, and encouraging fresh thinking. To that end, members must be willing to invest the considerable time and effort to develop deep expertise in national security, especially around the conflicts we are fighting.

Congress is also unique in its authority to peer through the cloud of secrecy that otherwise necessarily cloaks much of the conduct of war. This is all the more critical given the natural tendency of the executive branch under every administration—like any bureaucracy—to convince itself that whatever it is doing is working and that patience will ultimately vindicate the existing approach.

Advocates of a new authorization for use of military force sometimes argue that the authorization process itself—including some sort of periodic renewal mechanism—is the best guarantor for this kind of congressional overwatch. Unfortunately, this is mistaken.

On the contrary, past AUMF debates are striking for their failure to have anticipated the problems that arose in the conflicts they authorized. Current congressional deliberations around a new statute also don’t inspire much confidence in this respect: Those debates have focused thus far on procedural questions while neglecting the substantive issues about the wars the resolution would endorse.

Nor should intensive congressional scrutiny take place only every few years. Rather, it is a continuous responsibility for Congress that should be pursued independent from any AUMF mechanism. Members should be traveling and investigating, talking to experts, fleshing out alternative ideas, and working with executive branch officials and military leaders to improve the conduct of operations critical to our national security.

Too little of this seems to be happening now; late last year, after the combat deaths of four servicemembers in Niger, many legislators admitted they did not know U.S. troops were deployed there.

None of this is to diminish the constitutional case for passing a new authorization for the use of military force. But lawmakers owe our warfighters and our citizens more than an updated AUMF.  It is knowledgeable, constructive oversight of today’s wars that is the most impactful contribution that Congress can make to our national security—and failure to provide it, an even more problematic abdication of responsibility.

(Via Lawfare – Hard National Security Choices)

I tried to cut this down for citation. I failed. Read it all.