Tag: Project Reclaim

HOWTO: Ditch Gmail for self-hosted webmail

Update (January 2015): Although the self-hosted approach detailed in this post worked really well for the six months I tried it, I’m now using Fastmail.


Introduction

On Friday, I decided to ditch Gmail. It had been a long time coming, to be honest.

I’m not really interested in having a debate whether or not I ‘should’ do this, or whether it’s objectively a good thing to do. I just had an epiphany when I realised that almost all of my data (e.g. search, email, analytics) was going via Google’s servers. It’s like some kind of legitimised man-in-the-middle attack.

Instead of Gmail, I’m using webmail on my own domain and (shared) server. It was a pretty straightforward process. Here’s how to do it:

  1. Set up self-hosted email inbox
  2. Forward (and archive) email
  3. Import folders and email
  4. Update email addresses around the web

1. Set up self-hosted email inbox

I’m using Reclaim Hosting, which comes with something called ‘CPanel’ installed. This makes it much easier to install and maintain apps and services.

Sign up for a web host that has CPanel. Login, and go to the ‘Mail’ section of CPanel:

cpanel-email

Click on the ‘Email Accounts’ option. Fill in the email address and password – for example, I chose mail@nulldougbelshaw.com

cpanel-email-configure

You can now access your new email inbox by appending ‘webmail’ to your domain name – e.g. dougbelshaw.com/webmail. You have a choice of interfaces to access your email inbox. I chose Roundcube:

roundcube

2. Forward (and archive) email

The first thing you want to do is make sure that you continue to get the email sent to you in one place. To do that, you need to forward all of the email that comes to your Gmail account to your new self-hosted email inbox.

In Gmail, click on the gear icon and go to ‘Settings’. Once there click on ‘Forwarding and POP/IMAP’:

gmail-forwarding

Choose ‘Forward a copy of incoming mail to…’ and input the self-hosted email you set up in Step 1. You’ll have to confirm that account by clicking on a link that Google send to your new email address.

I chose to ‘archive Gmail’s copy’. It’s up to you what option you choose here.

3. Import folders and email

This is the bit that takes the longest. In fact, you can leave this running overnight and/or be doing Step 4 while this is happening.

Unless you’re declaring email bankruptcy, you’ll need to transfer your existing emails and folders from Gmail to your new self-hosted email account. Step 2 only redirects all new emails received.

First, you’ll need to use an email client like Mozilla Thunderbird (cross-platform) to download all of your Gmail folders and emails. To set it up, download Thunderbird and then launch it.

You’ll see something like this:

thunderbird-first

You’ve already got an email address, so click ‘Skip this and use my existing email’.

setup-gmail

Add your Gmail account first by entering your name, email address and password. Remember, if you’ve got two-factor authentication turned on for your Google account, you’ll need an ‘App password’ from your security page.

The default option is to connect via ‘IMAP’ which is what we want so leave it as it is and press ‘Done’. You should then see Thunderbird importing all of your folders and emails. This will take a long time.

thunderbird-gmail-folders

Note: ‘folders’ in Gmail are known as ‘labels’.

Now you need to add the self-hosted email account you set up in Step 1. To do this go to the menu and choose ‘New’ and then ‘Existing Mail Account’:

thunderbird-new-account

You’ll see the same box from when you added your GMail account. This time add your email address and password to the account you set up in Step 1.*

thunderbird-setup-self-hosted

Click ‘Done’ and you now have both your Gmail and self-hosted email account syncing with Thunderbird.

Now you need to select all of your Gmail folders/labels and drag them to your self-hosted account.

move-folders

Be careful to drag them onto the email address rather than ‘Inbox’ – otherwise the folders you drag will become sub-folders of your inbox rather than folders in their own right. Of course, you can always just drag them to ‘Archive’ if you don’t care.

Now wait. Possibly a very long time if you’re on a slow connection and/or you have lots and lots of emails and folders.

4. Update email addresses around the web

While your email and folders are transferring – and, to be honest, over the next few days/weeks – you’ll need to update your email address with the accounts you have around the web. There’s no great hurry for this, as your Gmail messages will be redirected to your new email inbox, but it’s nice to get things sorted.

You may also want to do one or more of the following:

  • send a ‘please update your addressbook!’ email to your contacts
  • use an email auto-responder on your Gmail account for a while
  • add a message about having a new address to your email signature

EDIT: Remember, if you don’t tell people about your new email address, your emails will still be going via Google’s servers (thus negating the point of the exercise…)

Conclusion

Did you manage to follow these instructions? Have you got a different/better way of doing it? I’d love to hear from you in the comments section below (or via this Hacker News thread)!


*Apart from believing in Open Source software (and working for Mozilla), one of my reasons for using Thunderbird is that it provides auto-setup for a much wider range of services than other mail apps. Also, the reason you see ‘SSL’ here is because I set up https on my domain using StartSSL. That’s outside the scope of this tutorial, but is also probably unnecessary if you’re planning to access your inbox via the webmail interface.

Banner image CC BY David Huang

Project Reclaim: a pragmatic update

Project Reclaim: September 2012

Boone Gorges, who originally started me on the path towards reclaiming my own data, has posted an update on his Project Reclaim. I thought it was about time I did the same.

I’d already mapped out the various tools I use (see updated version above), consolidated my blogs and experimented with openphoto.me. Just over a week ago I moved web hosts from Bluehost to Hippie Hosting. The latter is a co-operatively owned web hosting service.

There are, however, at least FIVE services that I’m not planning to move away from at the moment.

The first, and probably most obvious of these, is Twitter. At the moment it’s pretty much impossible to export my tweets in a meaningful way from the service. But a social network is nothing without community, so I’ll be there until something better comes along. (I don’t, however, use Facebook apart from a place to link to my newsletter or blog posts)

Next we have Dropbox. I really do love this service as it means I’ve got all of my stuff everywhere I am. We’ve also started using it within the Learning team at the Mozilla Foundation. I could try ownCloud or similar but when it comes to having my documents and sharing them easily with others, I want as little friction as possible.

Third is mobile-only social network called Path. All of my family now use this to post photos, videos and updates. It keeps us all in touch. Whilst it would be really nice to be able to get my data out of here quickly and easily the beautiful interface and benefits outweigh anything else at the moment.

Fourth we have Google’s services. With the proviso of switching off my search history and encrypting my searches, I’m fairly happy to use GMail, Google Drive and Google Calendar at the moment. Actually, that’s a bit disingenuous – they’re central to my workflow. I also use Google+ occasionally. Whilst I’m slightly uneasy with the situation, at least I can export my data quickly and easily using Google Takeout.

And finally comes Slideshare. I’ve experimented with using HTML5 slides but (a) they take while to put together, (b) I recently went over 100,000 views on Slideshare (one-hundred thousand!), and (c) I haven’t (currently) got the skills to make HTML5 slides as aesthetically pleasing as a a Keynote/Slideshare combo.

Conclusion

I’m going to continue with Project Reclaim as I think it gives me a steer in choosing the technology I use on a day-to-day basis. However, I’m going to be pragmatic about it. I could decide to use identi.ca instead of Twitter but, whilst that would give me a warm smug glowing feeling of openness, it would cut me off from contacts I find extremely useful and valuable.

You’ve got to balance these things out.

One thing that has been launched recently which is a huge boon to those looking to be more in control of their own data is ToS;DR. As TechCrunch pointed out, the biggest lie on the internet is that people read and agree to tortuously-long Terms of Service agreements. Instead, they’re more like “too long; didn’t read” (TL;DR). The Unhosted venture reviews Terms of Service agreements and rates them on various factors.

It’s definitely worth a look.

Project Reclaim: experimenting with openphoto.me

openphoto.me

As part of my ongoing Project Reclaim, and spurred on by D’Arcy Norman’s recent post on abandoning Flickr, I’ve been playing with openphoto.me.

In a similar way to the idea behind Unhosted, you bring your own data (i.e. photos) and the application does something with it (i.e. display them nicely, allow you to share them easily). I’m using Dropbox, but you can use Amazon S3, Box.net and more.

I like openphoto.me. I’ve uploaded two sets, one public and one private. The private one is of my children and shared only with family. The public one is of some photos I took down at Druridge Bay yesterday.

Where do you store your photos? Why?

Project Reclaim: consolidating my blogs.

I’m ill at the moment: I can’t seem to shake ‘flu-like symptoms that struck last Wednesday. On the plus side, not being able to do ‘productive’ work means I’ve got done some stuff I haven’t been in a position to prioritise for a while.

Black Heart Inertia

Posterous, a blogging solution I’ve really enjoyed using and have advocated widely, was bought by Twitter recently. It was a talent acquisition, meaning that the future of the service is in doubt. Yesterday, I spent some time moving my Conference and FAQ blogs (previously hosted on Posterous) to subfolders of dougbelshaw.com.

The next step is to find a way to transfer Thought Shrapnel, my Tumblr-powered blog, in a satisfactory way. Truth is, Tumblr is an excellent (although painfully proprietary) platform with some really nice features. I like the defined post types and the way you can queue-up blog posts to go live.

Another thing I’d like to do is move both this blog and my e-books space from separate installations to my new WordPress ‘multisite’ installation running on the site root.

Finally, I’ve discontinued blogging at literaci.es (transferring the posts here) and moved my Ideas Garden to a public Evernote workbook.

You can find all of these spaces linked to from my profile at dougbelshaw.com.

Image CC BY-SA Fey Ilyas


In addition, you may want to check out both Martin Waller and James Michie who have also been consolidating their online presence.

Beyond Elegant Consumption.

Beyond Elegant Consumption

At the Mozilla Festival last year, Mozilla Chairperson Mitchell Baker stood up and gave a short talk. Something she said really resonated with me. In fact, it resonated so much that I baked it right in as a central message of my TEDx Warwick talk.

We need to move beyond mere ‘elegant consumption’.

There’s nothing inherently wrong with elegant consumption in and of itself. Reading, watching and experiencing other people’s creations put together in a thoughtful and delightful way is joyful. But if that’s all we’re doing, then we have a problem.

I’ve championed Apple’s hardware and software since buying my first MacBook in 2006. I love the way that their offerings are so easy to use. At some point over the past six years I think I’ve owned or used pretty much their whole product line.

So why this week did I install Pinguy OS (a Linux distribution) on my iMac and trade my iPhone for the open-source Nokia N9?

Until last year, it was possible to swap out almost any hardware and software and still have a functioning ecosystem. An individual or organization could first decide what they wanted that ecosystem to look like and then invest in the constituent parts of that ecosystem. I feel like that’s changed. Now it’s a case of choose your vendor lock-in. And worryingly, that choice seems to be increasingly an aesthetic choice.

Yes, it’s nice that Apple, through iCloud, auto-syncs all of my stuff everywhere. And it’s wonderful that Google can present me with a (mostly) seamless experience on their combination of hardware and software. But I don’t want to have to buy into their whole ecosystem to get the functionality I require.

I’ll tell you what I want. I want interoperability. I want standards. I want a world where I can plug one thing into another and it (mostly) works. And if that world is slightly less shiny than it might otherwise have been? Well, that’s fine with me. At least I’ll have learned to start worrying and love my data.

Project Reclaim: backing up to local network storage

Learn about Project Reclaim here.

Netgear Stora

As, seemingly, most of the rest of the world, I’ve got a (50GB subscription to) Dropbox. I use it in place of the ‘Documents’ folder on my MacBook Pro and, at work, instead of the ‘My Documents’ folder in Windows. Everything is kept in sync between the machines and it’s all backed-up in the cloud.

That’s all well-and-good, and three places to store data is obviously a good situation to be in. However, given the recent Amazon EC2 outage (Dropbox uses EC2) I’d like to have a local backup solution. Until 2009 my wife and I used to do this with the use of an Apple Time Capsule, but the incremental backups used to slowdown our laptops so much that we eventually sold it. Every now and again I’ll backup to a 2TB external hard disk, but that’s only when I remember.

I wanted something better.

After looking at our needs and the options, I settled on a Netgear Stora* and two 2TB hard disks in RAID1 configuration**. This means that data is written to both disks simultaneously – i.e. a Redundant Array of Independent Disks. It came in at about £170 all-told, which isn’t bad at all – especially when you consider that it’s got secure web access to the files it contains and is extremely easy-to-use.

Once you’ve spent 10 minutes getting the Stora up-and-running, you need a way to get files onto it. That’s as easy as drag-and-drop if you want it to be, but I want a more robust solution. As with Dropbox, after the initial backup I only want to transfer the files that have changed. Enter rsync – or, more accurately, arRsync (Mac only). The graphical front-end is simple and effective. I refused to pay $40 for the privilege of the (admittedly widely-acclaimed) ChronoSync.


*Other NAS drives I looked at have bittorrent functionality. This can be enabled on the Stora by looking here or here.

**This isn’t a techie post, so if you want to read about RAID, I suggest this post on Wikipedia.

Project Reclaim: or, how I learned to start worrying and love my data.

The irony of a 'trailblazer' being rendered inert

A few days ago, via Stephen Downes, I came across Project Reclaim, an attempt by Boone Gorges to ‘reclaim’ his data from the multiple silos he’s been putting them in. He’s talking about those places where it’s easy to get data into but not so easy to get them out of: Facebook, YouTube, Twitter… the list goes on. I was in the right frame of mind to take action after listening to Jason Scott be interviewed recently about the importance of digital heritage.

To be clear: none of what follows is about getting a warm fuzzy feeling from being more ‘open’. It’s everything to do with having access to my data when I’m the same age as my parents. This stuff is important.

Eventually, I’d like to be running open everything, but the first step is to have control of my data. I certainly can’t trust Facebook to host my data, but that’s not to say it can’t be an output – a place where data from other places is mirrored.

The first thing I decided to do was to map as many places where I store things online. I highlighted those services in green that are based on Open Source technologies. Those in orange are services where it’s reasonably straightforward to get your data out. Those in red are those where it’s difficult to download and export your data.

Project Reclaim - mindmap

Over the rest of 2011 I’m going to be trying to make that mindmap greener. You can see that I’ve started to add (in pink) alternative services. I may have mis-coloured some elements (Dropbox should be orange, for example) – but the idea is sound.

Where am I going to start? I’m going to invest in Linode which means I’ll be able to host things like Sparkleshare (to replace Dropbox) and FlexPaper to replace SlideShare and Issuu. I’m not so concerned with the Google-based stuff at the moment as their committed (at the moment) to making exporting your data fairly painless – but I wish I could bulk-download my YouTube videos…

Image CC BY ecastro

css.php