User Tools

Site Tools


public:the_tale_of_httpdirfs

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
public:the_tale_of_httpdirfs [2019/04/30 08:53] fangfufupublic:the_tale_of_httpdirfs [2019/08/09 12:24] (current) fangfufu
Line 38: Line 38:
 It is kind of crazy how far this project has come - this software is now available on [[https://packages.debian.org/sid/httpdirfs|Debian]]. It is interesting enough to attract a Debian Developer who packaged and uploaded it.  It is kind of crazy how far this project has come - this software is now available on [[https://packages.debian.org/sid/httpdirfs|Debian]]. It is interesting enough to attract a Debian Developer who packaged and uploaded it. 
  
-Finally, researchers in Germany have decided to incorporate HTTPDirFS in their research software [[https://www.curious-containers.cc/docs/red-connectors-input-directories#httpdirfs|framework]], for importing data. Their publication record so far suggests that the project is primarily used for biomedical research. +Finally, researchers in Germany have decided to incorporate HTTPDirFS in their research software [[https://www.curious-containers.cc/docs/red-connectors-input-directories#httpdirfs|framework]], for importing data. Their [[https://www.curious-containers.cc/publications|publication record]] so far suggests that the project is primarily used for biomedical research. 
  
 I really don't know what to feel or what to say about this one - this project was originally designed to annoy someone on the Internet. It was not meant to be useful or helpful. It feels really strange that some researchers on the Internet are taking it seriously. Because I am in UEA Triathlon Club, I have a lot of friends who study medicine, I do enjoy being around them. But I find it highly weird that HTTPDirFS somehow winds up helping out with biomedical research - when will people who are somehow related to medicine leave me alone? (Only joking of course!) My dad does biomedical research, so I suppose it feels great to indirectly contribute to the field. :-) I really don't know what to feel or what to say about this one - this project was originally designed to annoy someone on the Internet. It was not meant to be useful or helpful. It feels really strange that some researchers on the Internet are taking it seriously. Because I am in UEA Triathlon Club, I have a lot of friends who study medicine, I do enjoy being around them. But I find it highly weird that HTTPDirFS somehow winds up helping out with biomedical research - when will people who are somehow related to medicine leave me alone? (Only joking of course!) My dad does biomedical research, so I suppose it feels great to indirectly contribute to the field. :-)
Line 45: Line 45:
  
 What is certain is that I am really proud of this project - it feels great that people on the Internet take your toy project seriously, especially when it wasn't meant to be serious at all. Using badly learnt knowledge learnt from undergraduate days in real life brought me great satisfaction. Thank you for teaching me about concurrency, [[https://www.cs.york.ac.uk/people/burns|Professor Alan Burns]].  What is certain is that I am really proud of this project - it feels great that people on the Internet take your toy project seriously, especially when it wasn't meant to be serious at all. Using badly learnt knowledge learnt from undergraduate days in real life brought me great satisfaction. Thank you for teaching me about concurrency, [[https://www.cs.york.ac.uk/people/burns|Professor Alan Burns]]. 
 +
 +===== Email to Professor Alan Burns =====
 +Race conditions [[https://github.com/fangfufu/httpdirfs/issues/38|might still be in my code]], because my code is crappy, and my knowledge is shoddy. I have emailed my undergraduate professor - hopefully he will give me some help. Hopefully, at least he would find my story funny -- I know at least if someone send me an email like this, I would love it. 
 +
 +<code>
 +Dear Alan, 
 +I don't know if you remember me. I was the 2nd year undergraduate
 +course rep back in 2012. I am currently a PhD student in University of
 +East Anglia. I am working on Computer Vision. 
 +
 +Thank you for teaching me concurrency programming in POPL back then. I
 +am afraid I didn't do so great in your coursework - I think I scraped a
 +40%. 
 +
 +However, the things you taught me has proven to be incredibly useful
 +and valuable, because my hobby project depends on it. I basically wrote
 +a filesystem (https://github.com/fangfufu/httpdirfs), and it is mildly
 +popular. Somebody in Canada decided to package my software and upload
 +it to Debian repository. Debian is one of the largest Linux
 +distribution. 
 +
 +Without the knowledge I gained from your module, I might not have been
 +able to figure out what I was facing. During the process of writing the
 +cache system for my filesystem, I have hunted down numerous race
 +conditions. It was kind of funny and bizarre when my code ran fine when
 +I forced it to run with one thread, but it failed mysteriously when I
 +decided to run it with multiple threads. I needed multithreading for
 +performance reasons. 
 +
 +I have since identified the critical sections of my code, and guard
 +them using pthread mutexes. During my own testing, I haven't encoutered
 +deadlocks. Unfortunately, my Canadian friend has reported a sympton
 +that sounds like a deadlock. The most annoying thing is that it is not
 +very reproducible. 
 +
 +I am pretty sure that if I persevere. I will eventually figure
 +something out. I am doing a PhD afterall. But I just thought my
 +situation is kind of funny, and you might appreciate the irony of my
 +story. I have never thought that my hobby project would depend on
 +knowledge from one of my worst performing module back in my
 +undergraduate days! 
 +
 +So my question is, us there a way to check for deadlocks by static code
 +analysis? I am sorry, but I do not remember if there was a way to check
 +if my solution to the dinining philosopher coursework would cause
 +deadlock, other than running it. It would be nice if you point me to
 +some reading materials. 
 +
 +Best wishes,
 +Fufu
 +</code>
 +
 +==== Professor Alan Burn's reply ====
 +<code>
 +Hi,
 +Thanks for your 'story', indeed ironic but interesting - I am pleased to have had a possible input.
 +
 +Deadlock detection is VERY hard - testing will not identify the subtle situations that can lead to this failure
 +
 +Two approaches - one use a resource usage protocol that prevents deadlocks (they exists for single processor
 +systems but are not as common for true parallelism) - two, use model checking on a model of your software
 +to 'prove' deadlock free in ALL circumstances. The latter is a powerful but not too easy to apply, especially to
 +code that already exists.
 +
 +Good luck
 +
 +Alan
 +</code>
 +
 +
 +===== HTTPDirFS was accepted into Debian repository by the then DPL himself! =====
 +Chris Lamb was the Debian Project Leader in March 2019. In his blogpost, he mentioned ''httpdirfs-fuse''. This is really cool!!! It feels like I unknowingly got an autograph from a Hollywood star!!!
 +
 +https://web.archive.org/web/20190731125517/https://chris-lamb.co.uk/posts/free-software-activities-in-march-2019
 +
 +
public/the_tale_of_httpdirfs.txt · Last modified: 2019/08/09 12:24 by fangfufu