Category Archives: annoyances

How can I automate repetitive find’n’replace operations?

I’m the webmaster (and a trustee) of the West Midland Bird Club, a registered charity.

Oystercatcher

Every month, I get sent a series of text files, with lists of bird sightings at each of our reserves, and some other locations. They usually comprise around thirty entries like these:

  • 6th: 1 Dunlin, 1 Oystercatcher, 2 Little Ringed Plovers, 1 Common Sandpiper, pair Shelduck, pair Greylag Geese flew over, 1 Cuckoo, pair Kingfisher, 2 Lesser Whitethroat at north end of Reserve.
  • 5th: 3 Oystercatchers, 1 Ringed Plover.

and I need to turn them into HTML markup like this:

<li class="hentry" id="D2011-05-06"><span class="entry-content"><abbr class="updated entry-title" title="2011-05-06">6th</abbr>: 1 <span class="biota bird"><b class="vernacular">Dunlin</b></span>, 1 <span class="biota bird"><b class="vernacular">Oystercatcher</b></span>, 2 <span class="biota bird"><b class="vernacular">Little Ringed Plover</b></span>, 1 <span class="biota bird"><b class="vernacular">Common Sandpiper</b></span>, pair <span class="biota bird"><b class="vernacular">Shelduck</b></span>, pair <span class="biota bird"><b class="vernacular">Greylag Geese</b></span> flew over, 1 <span class="biota bird"><b class="vernacular">Cuckoo</b></span>, pair <span class="biota bird"><b class="vernacular">Kingfisher</b></span>, 2 <span class="biota bird"><b class="vernacular">Lesser Whitethroat</b></span> at north end of Reserve.</span></li>

<li class="hentry" id="D2011-05-05"><span class="entry-content"><abbr class="updated entry-title" title="2011-05-05">5th</abbr>: 3 <span class="biota bird"><b class="vernacular">Oystercatchers</b></span>, 1 <span class="biota bird"><b class="vernacular">Ringed Plover</b></span>.</span></li>

to make pages like this one: westmidlandbirdclub.com/belvide/latest.

That involves a series of find’n’replace operations, in sequence, like:

  • Find Oystercatcher and replace with <span class="biota bird"><b class="vernacular">Oystercatcher</b></span>
  • Find Ringed Plover and replace with <span class="biota bird"><b class="vernacular">Ringed Plover</b></span>
  • Find Little <span class="biota bird"><b class="vernacular">Ringed Plover</b></span> and replace with <span class="biota bird"><b class="vernacular">Little Ringed Plover</b></span>
  • Find </b></span>s and replace with s</b></span>

…and so on. With well over 100 species in a typical series of reports, that’s a lot of faffing about. And it has to be done every month. It’s a right pain in the Wheatear.

I need to find away to automate this (under Windows XP, preferably GUI-based), working from a saved list of find’n’replace terms, and would appreciate suggestions. Is there a text editor with a facility for sequencing such operations? I could learn to write code to do it, but that’s a heavy up-front investment. Or would someone like to volunteer to help me put the code together?

Update: I’ve found a solution in ReplaceText which, though it’s sadly no longer supported and apparently doesn’t work under Windows 7, does just what I need.

Image of Oystercatcher in flight at Els Ness, Sanday, Orkney, by lukaaash.

Google Books: please make better use of screen space on my netbook

The screen-shot below shows the appearance of a 15 April 1972 Billboard magazine article about Pink Floyd, in Google Books, on the screen of my Lenovo S10-S2 netbook, using my default settings. 

Screengrab, showing the top of a scanned magazine page, with a picture and then only a few lines of text showing

Although I’ve used Google Books’ “full screen” setting, this only uses the full width of the screen, not its height. Of the viewport’s 509 pixels height, virtually a fifth, 98 pixels, are taken up with navigation tools which are, while I’m reading the article, redundant:

The top of the previous screengrab, showing navigation links

It would be good if Google would let me hide that header until I need it again, and thereby reduce the amount of vertical scrolling needed as I read the multi-column content.

The mysterious case of Birmingham’s missing Victorian cast iron urinal

My recent post about a metadata standard for syndicating information about public toilets, reminded me of an incident that occurred over 20 years ago, and questions about it, which have vexed me for years since…

In late 1986 and early 1987, to make way for the construction of the International Convention Centre in Birmingham, a number of older buildings had to be demolished, and entire streets disappeared.

West to East: St Martin's Place, Edward Street and Alfred Place, running parallel, between Broad Street and Cambridge Street

1949 map showing St Martin's Place, Edward Street and Alfred Place

On one of those, Edward Street, was a cast-iron street urinal. This was a listed structure: despite its mundane function, it was an impressive and decorative piece of Victorian engineering. The (then) Department of the Environment only gave permission for its removal on the condition that it was re-erected elsewhere. At the time, I was a volunteer at Birmingham Railway Museum, and so in January 1987 I wrote, formally, on behalf of the museum, with the approval of its management, to the City Council, offering a home for the urinal at the museum.

This request was refused by a council officer.

It was with some surprise, therefore, that we read in the Birmingham Evening Mail on 14 February 1987, an appeal from “Birmingham council chiefs” for a new home for the urinal:

Yellowing press clipping from 14 Feb 1987, showing the urinal

Birmingham Evening Mail, 14 February 1987

There were suggestions from members of the public that the urinal could go to the Black County Museum, or Avoncroft Museum of Historic Buildings — neither of which, unlike the Railway Museum, were in Birmingham, or had indicated any interest in the structure:

Yellowing press clipping from 21 Feb 1987, showing the 'Philip Colmore' column logo

Birmingham Evening Mail, 21 February 1987

I wrote to the Evening Mail, and to the Council, pointing this out:

Yellowing press clipping from 128 Feb 1987

Birmingham Evening Mail, 28 February 1987

Eventually, after reconsideration, the council’s Technical Services Committee (made up of elected members) agreed that the Railway Museum could after all have the urinal.

It was about this time that my ten years of volunteering at the museum came to an end, but I was told that the urinal had been dismantled and delivered to the museum, where it was placed in store.

Remarkably, the museum now tell me:

It is believed that the urinal was disposed to another museum … but I regret that at this length of time there is no record of where

So, where is our listed urinal — part of the City’s cultural heritage — now? Did the museum obtain — or need — the council’s or English Heritage’s permission to give up this object? Why were the public not consulted about the change of location, as they were the first time around? And did the council fail its legal duty to see that it was re-erected?

Update, 25 May 2011: One of my moles has suggested to me that the urinal may never have been delivered to Birmingham Railway Museum, and that it may even be languishing in a Birmingham City Council store. Are either of these things true? Let’s hope we find out. Meanwhile, the mystery deepens!

Update, 3 June 2011: The Birmingham Mail have written about this post, and there’s a Birmingham Post article about this post, too.

Update, 22 September 2011: Birmingham City council have just informed me that the urinal they have in store (referred to in my 25 May update) is not the one from King Edward’s Place, which their records say was sent to Birmingham Railway Museum. They are trying to locate the relevant paperwork for me. So the issues remain: where is our urinal, and was the legal requirement to reinstate it complied with?

Fixing Facebook’s Microformats (at their request)

Twitter, and the wider ‘blogosphere’, have been alive tonight (UK time), with people commenting on, or mostly simply repeating, the news that Facebook have implemented the on all their events, making them parsable by machines and thus easy to add to desktop or on-line calendars. They’ve also included for venue details.

This is generally a good thing, but what most people — at least some of whom should have known better — failed to notice was that the implementation is broken.

Consider this event, a concert by my friends’ band, Treebeard:

which, as you can see, is on 18 February from 19:00–22:00 (7–10pm).

That’s encoded, in the Facebook page’s mark-up, as:

<span class=”dtstart”><span class=”value-title” title=”2011-02-18T19:00:00-08:00“> </span>19:00</span> – <span class=”dtend”><span class=”value-title” title=”2011-02-18T22:00:00-08:00“> </span>22:00</span>

The “-08:00” at the end of each date-time value represents a timezone 8 hours behind UTC (as we must now call Greenwich Mean Time) — that would be correct were the event in California, or elsewhere in the Pacific Time Zone; but for the UK, the mark-up translates to 3-6am.

Since the event is in the UK, the start time should be encoded as “2011-02-18T19:00:00+00:00“, which puts it in the correct UTC timezone (in British Summer Time, it would be “2011-02-18T19:00:00+01:00“). Ditto for the end time.

The same will apply for events in any other timezone on the planet, each with an appropriate adjustment.

I’ve already alerted Facebook developer Paul Tarjan to the problem, and this is my response to his requests for assistance in fixing it.

Don’t confuse your social media channels

Earlier today, Birmingham‘s O2 Academy (a large popular music concert venue, m’lud) posted this to Twitter:

Tuesday’s giveaway….Black Rebel Motorcycle club CD’s to give away! Just head over to the competitions tab for more information!

Unfortunately Twitter doesn’t have a “competitions tab”, and neither do the various Twitter clients that people use.

As you can see from the suffix I’ve highlighted in the screenshot, “via Facebook”, the tweet they posted was actually a Facebook status update. It turns out that their Facebook page has such a tab, and the Academy have simply piped their Facebook statuses into Twitter, without thinking about, or remembering, what they’ve done.

A salutary lesson to be careful about feeding content from one forum to another; and about writing for a specific context. Failure to do so can give confusing messages, and is not helpful to your audience

An open letter to Facebook, about their broken microformats

Dear Facebook,

Thank you for adding an hCard microformat to my profile on your site.

However, it’s broken, as you can see in this screenshot, made using the debugger in the superb ‘Operator’ add-on for Firefox:

Microformat contains bogus "org" and "title" properties and no ""URL" or "email" properties

My Facebook profile, with broken hCard microformat shown in Operator toolbar's debugger

You need to fix some things:

  • I am not an organisation, so please remove the org property (you may have some user accounts for organisations, contrary to your own polices. That’s their, and your, problem — individual users are by far the majority).
  • The names of six of my friends, chosen by you at random, are not my titles. My title is currently “Mr”. (I say currently; it might change to “The Right Honourable”, if ever gets to be PM and I threaten to publish the pictures).
  • Add class="url" and rel="me" to my web addresses, This is probably the single most useful thing you could do for me right now. Unless you like ironing.
  • Add class="email" to… oh, you guessed, To my e-mail address; that’s right. I’m sure that won’t be hard to do.
  • Add a machine readable date and mark up my birthday as such: I might get more cards if you do.
  • Mark up my address as such, or at least as a label.

If you do this for me, I promise not to refer to you as “Farcebook” again. Until the next time you screw up, that is.

All the best

Andy
— x —

Open local public spending data – a potential hitch

There is — quite rightly, in my autocratic and what-does-humble-mean-anyway opinion — a move to have public bodies publish details of every item of spending over £500. I won’t go into the arguments about this, nor the technical issues, because that’s already been done by wiser heads than mine, Oh, OK, wise heads including mine (see comments on the latter document).

However, one thing in particular concerns me. Sometimes, a body — a local council, say, like, but not specifically, the one I work for — will receive grant funding for a project or activity. Such money usually comes with conditions attached.

Now, suppose this funding has two parts: £99,000 to do something which benefits the community as a whole, and £1,000 which must be spent on something seemingly trivial; say, publicising the activity by producing beermats. No beermats; no £99,000 to spend on a worthwhile activity. Such things do happen, if not literally demanding beermats.

Suppose that £1,000 spend is then published, along with hundreds of other items of expenditure. The finance office of the council will not know about the grant funding, or the conditions attached to it, nor do they need to. They will just add an entry to a database, saying “Acme Beer Mats, 1 April 2011: £1,000”, which will then be made available with all the other enteries in that database, as open data

Along come the Daily FMail and the Taxpayer’s Alliance, and before you know it, the media and (ironically, given my frivolous example) bar-rooms up and down the country are full of “Borsetshire Council wastes money on beermats extravaganza on the rates[sic]” headlines.

No doubt the authority will put out a subsequent press statement pointing out the £99,000 of benefits, the unavoidability of the attached conditions, and so on. And no doubt it will receive little if any press attention.

What can council’s do to prevent this scenario? Annotate every spend item in their published data? Surely impractical. List such items separately? I don’t know (and don’t get me wrong, I’m an open-data advocate; and this is a relatively minor matter, which shouldn’t stop such data from being published), but do I hope somebody has an answer.

Over to you…

Footnote: thanks to for encouraging me to blog my rambling comments on this, made during our earlier discussion.

More police forces should publish web pages about wildlife crime officers

I maintain the West Midland Bird Club website. The club serves the four English counties of Staffordshire, Warwickshire, Worcestershire and the Metropolitan West Midlands, and so I wanted to write a little bit about the work of the relevant police services’ Wildlife Liaison Officers (WLOs), and to link to web pages about each of them.

I was surprised to find that none of their websites have a page about their WLOs, or their approach to wildlife crime in general. So I started to look at other forces’ sites, and found very few of them did.

What I Wanted

I decided it would be a good idea to collate a list of the few good examples that exist and conversely the forces which don’t have such a page, not least to encourage them to provide one.

A quick Google search showed that no such list is already in place, so my next step was to check — on Twitter, of course — whether anyone else was working on one. The answer was no, but two projects have more generic lists of police websites and related details in hand: OpenlyLocal and Podnosh.

I decided that I would have to make the list happen for myself, but I wasn’t going to do all the work. of OpenlyLocal kindly offered use of his data on police forces and their home pages and other contact details — it’s open data, under a CC license. That was in XML, though, and I lack the skills to manipulate it, so Chris kindly and quickly provided a dump into CSV format, suitable for use in a spreadsheet. A salutary lesson, there, to anyone publishing open data. While RDF and linked data is the way to go, so that it can be parsed and processed by machines in an sophisticated way, making a format like CSV available as well opens that data up to less technically-gifted users.

How I Did It

I copied the relevant columns from Chris’ document into my own, and made the editable spreadsheet available in Google Docs, for anyone to edit. I then blogged about it, inviting people to help me to full the missing column of wildlife crime pages. I was quickly retweeted by several people and organisations (thank you for that) and, significantly, the initiative was mentioned on the Guardian website by . This resulted in spate if activity, with most of the rows completed within a day or so after the mention. Interestingly, the spreadsheet filled up from the top, so it seems that unknown volunteers were helping with the first gap they came to, rather than that for their local force, which is what I had expected.

A few people shoe-horned prose comments into the URL column, so I added a “notes” column and moved their comments there.

After a few days, only a couple of gaps remained, so I filled these myself, and locked the spreadsheet prevent vandalism (any amendments may be posted below, as comments).

Findings

So, whet did we find? Of 51 forces, only 29 have a wildlife crime page — and some of those are patchy. Other forces don’t have one, but mentioned the work of their WLOs in press releases, progress reports and policy or strategy documents — sometimes in PDF files.

In some cases, a search of the force’s website for “wildlife” returns no result at all — a disappointing state of affairs.

PC Duncan Thomas, Wildlife Liaison Officer with Lancashire Constabulary

On the other hand, there were some great examples of best practice, including Lancashire Constabulary, Merseyside Police and North Yorkshire Police, from which other forces can learn. Note that they variously make use of video, and have links to wildlife conservation bodies.

Also of interest is this article about the work of WLOs in Sussex, reproduced on Coldean Residents Association’s hyperlocal site.

What next

This is what I hope will happen now:

  • Each police force should set up a locally-relevant web page about wildlife crime and their response to it, with relevant contact details, modelled on the best practice we found.
  • These pages should have short, permanent URLs so that links to them will not decay when forces change their technology
  • A central police website could ask an enquirer their postcode or address (or simply geo-locate their browser), and type of concern, then return the relevant page (whether it’s about wildlife crime, drug dealing or lost property) using the model adopted by LocalDirectGov
  • Websites listing details of all UK police forces’ details — like OpenlyLocal and Podnosh — could include their wildlife crime URLs
  • Wildlife websites with pages for each county (for example, Fatbirder, BirdGuides etc) could include the relevant forces’ wildlife links.
  • Local wildlife organisations (Wildlife Trusts, county bird clubs, RSPB Local Groups) should link to their local forces’ wildlife page

How you can help

  • Publicise this blog post and the open data that’s been genreated
  • Make use of that data
  • Write to your local force, if they don’t already have a page, and ask them to provide one — feel free to send them the URL of this post
  • Ask your councillor to encourage the local force to do so
  • Ask your local Wildlife Trust, bird club or related organisation to do the same
  • Let everyone know about the results, in the comments below, or with a pingback from your own blog post

I’ll notify national organisations like the RSPB, RSPCA and British Trust for Ornithology.

Meanwhile, if you wish to report wildlife crime in progress, call 999, or otherwise report it to Crimestoppers (who will treat the report as anonymous if you wish) on 0800 555 111.

Thank you

Finally, thank you to everyone who’s contributed to this project, to date.

Updates

20 August 2010: Warwickshire Police’s wildlife crime page was missed, because it was on a separate site, where their site’s search didn’t find it. It’s now been moved to the main site, as a result of this post.

3 September 2010: Northumbria Police pages created in response to this campaign.

June 2011: Staffordshire Police page created in response to this campaign (per their e-mail).

Google Maps’ microformats: unhappy anniversary – still broken after three years

Three years ago today — on 31 July 2007 — Google proudly announced that they had added hCard microformats to Google Maps, so that, as they put it:

your browser can easily recognize the address and contact information in the page, and help you transfer it to an addressbook or phone more easily

Less than four hours after seeing a mailing-list repost of that announcement, by Google‘s Kevin Marks (one of the two signatories of the initial announcement), I replied, pointing out that the implementation was badly broken, and that none of the microformats in a search for a single entity, in this case a school, were valid. (As is usual on Google’s own blogs, there was no facility for comments on the original announcements.)

Google‘s Gregor J. Rothfuss, the announcement’s other signatory, replied that he would look into the matter.

Almost a month later, I asked Gregor if there had been any progress, and he said (I quote him in full):

i will work on it when i have some time.

so I took him at his terse word, and left him to it, with no further reminders. That’s the last I heard from anyone at Google on the issue.

Three years on, though the specific faults have changed, not one of the microformats in the Google Maps search linked above is valid (the mandatory “fn”, or “formatted name” property is missing; address components lack the mandatory child-properties) and I have been unable to find one that is, in other results. They are as useless to someone wanting to add the subject’s address to their address book today as they were on day one.

Update, 31 July 2011: Another year has passed, the microformats are still broken.

The Highway Code should be available as a set of linkable HTML documents, not just PDFs

The Highway Code is:

the official road user guide for Great Britain ()

Drivers must study it in order to pass a driving test, and all road users should remain familiar with it — including any revisions — throughout their lives.

It’s available online, but large parts only as a series of large PDF files. That means that when, for example, my friend Pete Ashton asks:

It’s still illegal to park on double yellow lines, right? on Twitter

I can’t easily answer him by linking directly to the section of the Road Markings PDF which says:

Double yellow lines mean no waiting at any time, unless there are signs that specifically indicate seasonal restrictions.

I would like the Highway Code to be fully available as a series of plain old semantic HTML web pages, with each section within each page having a unique ID (which even the parts already in HTML currently lack), so that I can link to the relevant, specific, section when I want to refer to it. For example, the section quoted above might be at http://www.direct.gov.uk/en/TravelAndTransport/Highwaycode/markings#double-yellow

Dear Government, Can you do that, please?

Update: I’ve asked my MP, Khalid Mahmood, to do what he can to assist with this request. I’ll let you know what happens.