Using Live Mesh and Git – the best of both worlds
git, internet, live mesh, microsoft June 4th, 2009I like Live Mesh. I like Git, too.
If you’ve used a folder synchronizing tool before that will propogates file changes between machines, like FolderShare or DropBox, there’s a good chance you’ve had a scare with some possible data loss. Maybe you accidentally dropped a folder into a subfolder, “made space” by getting rid of files on one of the machines, or the sync software went crazy. Whatever the case is the change is immediately shipped onto all of your backups all over the world.
That effect makes file sync tools more of a convenience than a backup strategy – it can lower your mean time to recovery in case of hardware failure – but even though there are two or more physical copies of your bits there’s really one logical copy that appears in several places where it can be accidentally deleted. Ironically the convenience of having your data everywhere also lowers it’s mean-time-to-accidental-destruction.
Which is exactly the problem a change control repository like Git is designed to solve! Fearing data loss as much as the next person, I tried to combine the convenience of Live Mesh with the safety of Git and it worked really well. Here’s how it went, assuming you have msysGit and Live Mesh installed on your machine.
Update here’s a diagram of what we’re ultimately going to try to accomplish.
Initialize Git
First off – the goal is not to put a Git repository into a Live Mesh folder. Git already has a push-pull mechanism to move changes around and the word is Live Mesh will just trash your repos files by moving overlapping changes over each other on different nodes. Instread what we’ll be doing is creating a new Git repository and using a Live Mesh folder as it’s working copy.
:create a new folder to hold files mkdir "%userprofile%\Documents\Junk Drawer" :go to it cd "%userprofile%\Documents\Junk Drawer" :initialize a repository git init :mark the .git subfolder hidden attrib +H .git
Or if you prefer to avoid command-line you can do the same thing with Git GUI and Explorer.
Initialize Live Mesh
Then the magic happens – add the Junk Folder to Live Mesh.
Throw some files in there and they’re automatically synchronized onto Live Desktop online (and to other devices which you have synchronizing the Junk Drawer). The Live Desktop gives you a browser-based Explorer-like interface to your files.
You can now access and modify the files in the Junk Drawer from any number of devices, including downloading and uploading files through mesh.com. All changes will be synchronized at every location including the one we originally created which is the working directory of the Git repository.
Commit Changes
Git keeps track of your files in the working folder when you select the changes to stage, provide a comment, and hit commit. Because it’s all synchronized it doesn’t matter which device caused the changes. There are plenty of Git tutorials out there so I won’t go through all the steps of adding files and committing them, looking at history, restoring files from source control. In the end the point is that this Git repository is now providing change control tracking for my Junk Drawer folder and (by extension) all devices on the Mesh hosting that folder even if they don’t know it.
Push Changes
If we stopped here there would be a history and some protection against accidental deletion (because you can get the files out of Git if they disappear) but there’s still a risk we’re exposed to. All of the repository data is saved in the hidden “Junk Drawer\.git” folder which could be lost if the drive fails or if the Mesh goes insane and wipes out the entire Junk Drawer parent folder.
Fortunately it’s a really easy problem to take care of – from this git repository simply initialize a new remote repository and push all changes to it. Okay, maybe it’s a little tricky to get set up because msys and git are a bit… awkward… when it comes to drive letters and such. So here are the commands you’ll use to create a remote repos on a known UNC.
:map a drive long enough to initialize an empty repository. :use whatever unc and drive letter is appropriate for you net use G: \\10.0.0.3\storage :go into the working folder cd "%userprofile%\Documents\Junk Drawer" :and create a bare repository on the file server :use whatever folder name makes sense :the foldername.git extension is conventional for Git repositories :that don't have a working copy of the files checked out git clone --bare . G:/junk.git :unmap the drive letter net use /delete G: :inform the local repository that "origin" refers to this new unc git location git remote add origin //10.0.0.3/storage/junk.git :and finally push over to the "origin" repos everything on the default "master" branch git push origin master
Once the association between the local and remote repos is established you can push to it from Git GUI any time you commit changes on your mesh.
Conclusion
And there you have it!
* You can have a Live Mesh folder synchronized between any number of devices.
* Each device can create, access, and modify those files normally without being aware of change-control
* You can commit changes on your mesh folder into a Git repos exactly as if it was a working folder
* You can push commits to an even more rugged Git repos on a distinct and isolated storage location
And now your files are safe – even if Live Mesh goes completely insane destroying all of your files and the hidden .git folder inside the original device’s Live Mesh folder. All you need to do is clone the Git repos down from the storage location back onto your hard drive and you’ll have all of the changes you pushed earlier in a new local repos and a rebuilt working folder filled with all of your files. Mark the new .git folder as hidden again, add the working folder back into Live Mesh, and you’re up and running again.








June 4th, 2009 at 7:48 am
Cool beans! Did you just build a distributed 2-phase commit for source control? I’m always worried about the unpredictability of the file syncing tools. FolderShare and Mesh have both surprised me in the past with file updates.
Also, just to be clear, this approach is only suitable for single-user syncing right? And it should work with an old-school tools like SVN as well.
June 4th, 2009 at 8:19 am
I haven’t tried it with multiple users on a folder, but I think it should would work exactly the same. Someone else makes a change, it appears in your working copy, then it shows up in Git GUI when you go to commit changes.
You could try it with SVN but the results probably wouldn’t be as good for mechanical reasons. You’d need to mark a .svn folder hidden in every subfolder of the mesh, and if subfolders were moved or renamed you’d have the classic tree-layout-mismatch situation between your working copy structure and the SVN repos urls.
I could be wrong, but I suspect SVN would cause enough headaches to neutralize the benefit.
June 4th, 2009 at 8:33 am
[...] Using Live Mesh and Git – the best of both worlds (Louis DeJardin) [...]
June 4th, 2009 at 11:25 am
Nice post. I have tried something similar, and it works really well. Check out my approach to this on: http://blog.fossmo.net/post/Ten-steps-on-how-to-store-Git-repositories-in-Live-Mesh!.aspx
June 4th, 2009 at 12:24 pm
At first I thought this was an abomination, but I can see the benefits. Dropbox versions your data, though, so there’s little chance of accidental deletion there either.
June 4th, 2009 at 6:38 pm
Be *very careful* with doing this – I lost two repositories because Mesh doesn’t handle merge conflicts well and will trash your .git folder.
Make 100% sure that all your files are synced on one machine, before you use a different one (i.e. using this on a laptop and a desktop is a BAD idea).
That being said, when it does work, it’s awesome – you just have to be careful.
June 4th, 2009 at 7:29 pm
@Fossmo/@Paul – well, to be clear in this example there’s a difference in that the Git repos is //not// cloned between devices. Only the working-copy files are moved around. You can see that in the browser screenshot of “Junk Drawer” there’s no .git folder synced with the cloud.
That’s why it’s very important to mark the .git folder “hidden” to begin with. Hidden folders won’t replicate, so Live Mesh will turn a blind eye to the repos and won’t end up chopping the child in half with conflict resolution.
@Gabe excellent use of the word abomination! Love the metaphorical use of morality applied to technology: the server is happy, that code is evil, this technique is an abomination, etc.
June 8th, 2009 at 10:54 am
Really very nice post. i’ll try this.
June 12th, 2009 at 12:24 pm
I’m not sure I understand the problem you are solving. Let’s say I had a hosted subversion repository, and I added the folder “Junk Drawer” to it. Then I pulled it down to “low cost small notebook pc”, I then have a way to get latest changes. Granted I assume live mesh does not require you to “check in” your changes, so SVN has a manual step, but this same step will be required to check into git. Also SVN will put ugly .svn files all over, but it seems to solve the problem.
This obviously is not a solution for average pc users, which Live Mesh is intended to be. But let’s be honest, git isn’t either, and it is not the audience of this blog.
At first glance, this to me seems to be an over-engineered solution. You’re a smart guy, and sometimes that’s what smart guys do. But at this point, I think I just don’t completely understand the problem you are solving.
I also keep somethings on a SVN repo on a USB Stick and carry that around, works well for versioning and portability because I don’t have to be connected to get the latest version.
June 12th, 2009 at 2:38 pm
Well, yes that’s true. If you’re using SVN or Git with normal source control patterns there’s no reason whatsoever to weld Live Mesh onto the working folder. That’s not the direction this is coming from.
Coming from the other side, starting with Live Mesh, it has a lot of functionality you won’t have from a change control repos. Instant replication, mom-compatible read-only invitation to photos, use of mesh on windows mobile devices (without out svn/git mobile port), other device support which may come out eventually like xbox, etc.
So the goal is to take that baseline and tack change control onto it.
If you want to talk over-engineering this isn’t even scratching the surface. :) Let’s see… Engaging imagination…
How about this: get your hands on a copy of the Mesh SDK and create a new device type which acts as a “bridge” to the svn or git repos of your choice. That way “GIT” or “SVN” would show up as a device in your ring, you could choose which folders you want to synchronize, and it could automatically keep that folder in the repos current preserving history as it goes.
If you accidentally delete content, connect to the repos with your favorite scc client app to get it back from history and drop it into the mesh folder to restore it.
June 12th, 2009 at 2:55 pm
That makes sense. The more I thought about it after commenting the more I realized this was filling a gap left by Live Mesh, which I don’t know much about, which left me not really understanding the problem.
Thanks for the reply.
P.S. I look forward to your Live Mesh SCC Bridge ;)