[Freedombox-discuss] Should we change the mailing list format?

[Paul D. Fernhout]

On 5/9/11 10:13 AM, Chris Troutner wrote:
> Hey all,
>
>
>
> I'm sure that I'm not alone in getting swamped by emails from this list.
> It's great that the list is so big and so active, but the meaningful
> emails containing important information/thoughts tend to get swamped out
> by the enthusiastic but less meaningful emails.
>
>
>
> Would it be a good idea to move this mailing list to a different format?
> I'm thinking of something that would combine a mailing list with a
> feature like Digg where people could have some way of voting on or
> 'like'ing a submissions so that the most meaningful (as determined by
> everyone) rise to the top.
>
>
>
> What do you all think?

In general, I agree with Daniel. With that said, here is something I 
wrote a couple months ago but did not post to the list then.

I doubt anyone will read it through (tl;dr etc.), but you can think of 
this in terms of thinking about what should go into a FreedomBox to make 
it truly easy to use for messaging. :-)

===

On 2/24/11 10:49 PM, Thomas Lord wrote:
 > I have a narrow topic question ... not the usual
 > "big picture" stuff.  Anyone want to help
 > brainstorm about:

I wrote this about a week ago, but I guess I might as well post it for 
future reference... It's about bugs in email handling tools at several 
different levels. This from someone who has been using email for more 
than a quarter century...

Note this is not a criticism of FreedomBox using GNU Mailman, which is a 
good system. This is more a criticism of email as a discussion platform...

=== Exhibit A: Obtaining the FreedomBox Email Archive

This is intentionally long and confusing. :-) It is also from memory 
mostly after the fact, so I might have left something out or got a minor 
aspect of the sequence wrong.

OK, so I wanted to import the FreedomBox email archive into Thunderbird 
from:
   http://lists.alioth.debian.org/pipermail/freedombox-discuss/

That was after signing up for the list, including mistyping my email 
with a colon before the @ the first time and getting an error message.

Now, this is what happened on BSD-based Mac OS X, but I expect it would 
be more or less about the same on Debian. Yes I used to run Debian on 
custom self-assembled hardware for about five years before switching to 
a Mac Pro couple year ago (which is usually blissfully quiet and just 
always seems to work right more or less), and maybe I will go back to 
Debian again on my primary machine someday, for now I run it in 
VirtualBox. Personally, I preferred Debian as far as GUI (one menu bar 
is a stupid idea), but my wife, and then a year later, I, got tired of 
all the random breakage during upgrades back then, and never being able 
to get the power management to work right -- issues perhaps since 
corrected. Otherwise I use mostly FireFox, Thunderbird, and Eclipse, and 
use languages like Python and Java, so most of what I do is identical on 
both platforms thanks to free software.

* First, I had to understand I could download archives and put them in 
Thunderbird. I've done this before several times, but it might not be 
obvious to many people they could do this at all.

* I clicked on a download link in Firefox. A file was downloaded, but I 
was not sure where it happened so fast (I usually do save as, but I 
clicked on it that time). Usually it is saved to a "Download" directory 
I have hundreds or thousands of files in by now (if you include saved 
web pages) but sometimes I download to other directories and FireFox 
remembers the last one. Ah, I know I need to find the first one, so I 
just click on them all now as I've already paid a price in having to look.

* So, I go looking for the files. I don't know what they are called, so 
I go back to the web page and hover over a link to find one is called 
"2010-September.txt.gz". I search for the beginning of that text, the 
Finder defaults by searching for contents so I switch to search by name, 
and I find two files with that name. So, I figure out one is the recent 
download by looking at the date and remembering I had previously 
downloaded the p2presearch archive files when they switched servers, 
which had one with that name from the same time. Note, all the context 
of where that file came from or what its purpose is, or what the name 
means as a date, is lost with the download process, even if there might 
be some records somewhere on my machine (Firefox?) of where I downloaded 
the file from. I create of folder, FreedomBoxDownloads to move all the 
files to. Now, I have so many files in my download directory, it takes a 
good long while to figure out where the exact files are. And I have 
other files named starting with 2010 in there, so it takes a bit of care 
to pick out just the right ones. But, I think the right ones are now in 
that subfolder. (Yes, I should have made a subfolder before I 
downloaded.) OK, now I need to click on them and unzip them.

* OK, now I figure out where to copy the files to put them in 
Thunderbird. Is it Library or Documents on the Mac that has my data for 
different things? After two or three years back on the Mac -- (I used to 
own a Mac Plus and other pre OS-X stuff and more than thirty years of 
using all sorts of different systems) I still hesitate about that 
because "Library" is usually something you go to for stuff other people 
have written and is permanent. So I search on Thunderbird, but it 
defaults to searching by contents again. Switch to "by name", find 25 
matches or so, but none look right. I know I should not put the data in 
the Thunderbird.app folder, but mainly because I've developed those 
things and know from previous experience Thunderbird stores stuff 
elsewhere. There is a "thunderbird" folder with an initial lowercase 
letter, but it is unrelated from a previous download. So, a bust.

* So, I go looking under Library, and find a Thunderbird folder. Why it 
did not show up in my previous search, I don't know. Oh, looking now, I 
see I was searching "This Mac". When I click on "Library" it finds it. 
Now, that does not make any sense to me at the moment given you would 
think searching on "This Mac" would find more stuff. Is my Spotlight 
desktop search database system corrupted? Anyway, so, I found the folder 
by tracing through the folder hierarchy from my user folder.

* OK, so I want to drag the mailbox archive files into the Thunderbird 
folder somehow. But where were they again? So, (fuzzily remembering 
exactly) I open a new Finder window, find the files again in Downloads 
and that subfolder. I make a folder in Thunderbird in the right place in 
the subfolders after more navigating past my funny UUID for my 
Thunderbird user data (there are two, as I copied the original default 
one before migrating my mail from Debian). I know I'm going to need to 
copy individual emails over from each individual archive to merge them 
into the FreedomBox folder because I have done this before.  So, I make 
a FreedomBox folder in Thunderbird. I also make a FreedomBoxDownloaded 
directory to copy the other files to to start working from. So, anyway, 
files copied, finally. I hope. Let's try to access them in Thunderbird.

* Now, I wont go into the details, and despite having done this several 
times before, after closing and opening Thunderbird several times, 
trying a variety of menu options, looking at the Import menu, and 
renaming files, changing extensions, deleting derived ".msf" files, 
moving files, and so on, basically experiencing a lot of frustration, I 
can't get Thunderbird to pay attention to these new mbox files with a 
".txt" extension (or even without one). I know I've done this in the 
past and it worked (but never understanding 100% why it was tricky). But 
some of that maybe was with the Debian version of Thunderbird? Or just 
older versions of Thunderbird? I know gnumailman uses mbox format which 
Thunderbird also uses (or I think I know that). So, anyway, fifteen 
minutes later or whatever, I still can't get it to work. But I know from 
thirty-something years of working with computers that I can usually get 
things I've done before to work again if I try hard (and sometimes I can 
even get stuff to work that I have not done before :-), so I keep trying.

* A couple times I look at Google for help, and get to these pages 
eventually mentioning ImportExportTools (MboxImport enhanced):
   http://kb.mozillazine.org/Importing_folders
   http://www.nic-nac-project.org/~kaosmos/mboximport-en.html
Now, OK, maybe something changed on Thunderbird with new versions and 
something and I need this? But, I don't know, that site is in Italy, and 
while I'm hoping that Italian LENR (cold fusion) stuff works out, do I 
really want to install an piece of software that says it does "auto 
updates" in my most central tool (Thunderbird) just to do this one task? 
Is it going to ask me before it updates? I'm not sure. I look at 
Thunderbird add-on window to see if it is in the standard list. I search 
for it. Nope, it is not there. So, the Thunderbird project is 
recommending a key piece of software no one can bother to put in that 
list. Is this thing for real? Is it reputable? Should I download it? Do 
I even really need it when I have not needed it before? Well, I've had 
some good luck with software from Italy before, and used to interact 
with people at IBM Vimercate it Italy when I was contracting at IBM 
Research helping develop embedded speech hardware, so, what the hey, I 
figure most of what I email gets scooped up by Carnivore etc. anyway, 
and why should some person in Italy care much about spoofing my email 
and taking control of my desktop? But, how good is their security to 
keep someone else from pushing malicious updates? So, OK, I'm accepting 
now someone in Italy (or someone who breaks into their computer) is 
going to be able to do anything they want to my email or maybe my 
computer. OK, I guess I should not be too paranoid or I'll never get 
anything done -- there is always a tension between security and 
effectiveness. There is a big wall of text on the download page I don't 
really want to read right now about how  to use it and lots of changes 
-- is this thing buggy? Or is it well maintained? So, I go ahead and 
click on the link to download the .xpi and Firefox is warning me at the 
top of the page about installing software from the internet. Maybe it 
thinks I want to modify Firefox? I'm not sure. So, I more carefully 
right-click over the link and save it. It gets saved, somewhere.

* Now I have to install it in Thunderbird. OK, where was it, Downloads 
directory? Yes. The install worked, and I'm staring at my last option to 
cancel the install. But I go forward and restart Thunderbird. Now I have 
to figure out how to use it. Where did it add itself to the menus? 
Search, oh there it is, I think. I pick Import, but it is the old Import 
window that did not work last time. It's at the bottom, 
ImportExportTools. But I go there and import is greyed out. Now, how to 
get it active? Well, maybe it has something to do with what window I'm 
looking at in Thunderbird. I bring up the main window with my email 
archive, pick the top in the hierarchy for my email, and yes, now import 
is active. So, I go through another dialog box (which option to pick 
about individual files and directories?) and now I'm navigating back to 
the Downloads folder to the directory. OK, is seems to have imported the 
files. I look in the folders, and yes, there are the FreedomBox emails!

* But, I kind of want them all in the same folder. So, I want to copy 
them over. I know from previous experience that sometimes the order you 
select things matters in a copy. So, I use select all to select all the 
emails in each month's archives (hoping it selects them in the right 
order). Then I move them into the FreedomBox folder I previously made. I 
do that for each month. Now I can delete those empty monthly archives. 
So, I have all the email.

* Let me check it out. I sort by date. Hmmm. The first email, by "might" 
has an empty subject and a date of 12/31/69. Did something go wrong? Or 
is it just that person's computer clock was messed up when sending the 
email? Maybe it is spam and I should delete it? The rest look OK. So, I 
look at that message and it looks like a reply to something else. OK, I 
don't know what month of the list that item was really posted to, or 
what the subject connects to. I guess I could figure it out, but Ill 
just ignore that for now. Even if I did know the right subject or the 
right date, I'd have to edit the text file of the mbox archive to fix 
it. It's just not worth it to me. So, I'll try to ignore it even though 
it is going to probably be sitting there annoying and nagging me forever 
now, every time I look at the archive at the top.

* OK, I go to the first email posted to the list with the title "Let's 
make the Freedom Box!" by "Tom Marble". I had read it using FireFox 
yesterday already. I know the web showed a reply nested under it 
"couchdb and couchapp" that I thought informative by "Ged Wed". I need 
to paste it in here as a link to a central server, not some UUID:
   "[Freedombox-discuss] couchdb and couchapp"
http://lists.alioth.debian.org/pipermail/freedombox-discuss/2010-August/000001.html
But, that email doesn't show up nested. Oh, yeah, click on the little 
spiral icon to nest the emails by topic. Still not nested even as other 
stuff is netted into threads. I guess the two systems (Gnu Mailman and 
Thunderbird) disagree about how to associate emails into threads. OK, 
well now I can read the rest of the emails. It works!

* Then I write this email. The good news is, I have a treadmill 
workstation, so at least I've managed to walk 0.96 miles over the course 
of over 100 minutes while doing all this and writing this email. So, at 
least I got some basic exercise in.

=== After all that

Now, how many people are going to be willing to do all that, to begin 
with? Or even think to do it?

Afterwards, I did find my subscription email amidst all the thousands of 
spams in my Inbox (sometimes lying about their timestamps into the 
future), and finished subscribing to the list.

I realized some messages had posted from when I downloaded the archive, 
so for at least one file, I had to go through and do the whole latter 
part of the process again above (download an archive, where, rename it, 
unzip it, import, carefully compare messages, move some, delete the rest 
of the archive). Oh, and now that I did that, it turns out the archive 
does not have stuff from today in it (I'm guessing, maybe it creates 
those files in a cron job once a day?) so it that was pointless. So I 
still can't get copies of the recent email messages and will have to 
read them through the web interface (giving up what privacy about what I 
look at to whom?). I'll have to remember to try to download the archive 
again tomorrow [this was written about a week ago, and I never did].

And next I have to set up a filter in Thunderbird to direct FreedomBox 
posts into the archive (after I figure out what is unique about them 
after I find them in the spam in my inbox -- is it sender or bracketed 
subject item perhaps). Of course emails to me may not look exactly the 
same as emails downloaded from the MailMan archives as far as headers, a 
minor thing, but it bothers me, nonetheless as an inconsistency. Also, 
should I group the meta messages from list subscription in with my 
FreedomBox folder, or put them somewhere else?

I know, someone is going to say use Gmail, or someone else is going to 
ask why I want a copy of the archives and why don't I just read the 
stuff on the web and link to it there, or someone else is going to say 
this old fart's memory is going that he can't remember where some file 
is of the literally millions on his hard disk, and it might all be true. 
:-) But the point is, this is what someone who has been using technology 
for a long time goes through to do the most seemingly basic thing to 
read some posts by those advocating decentralization and redundancy 
locally in a decentralized way. That is something you would think that 
could be accomplished with one click and menu choice (connect to remote 
semantic archive?) or, at most, a click and a drag. This also shows the 
fundamental problem of dealing with deeply entrenched social standards.

Sure, people can fix this specific problem, and scratch this specific 
itch. But, there is some fundamental problem going on here that all our 
existing tools are not solving related to social media and social 
computing. Putting Debian on a wallwart does not, by itself, address 
this deep issue. It only makes it worse in a lot of ways, by embedding 
these problematical standards everywhere.

IMHO, we need better tools and standards for communication and 
sensemaking for complex topics. Stuff like a semantic desktop
and tools built on top of it.

Even after I've succeeded at all this, what have I got but a bunch of 
emails I can't easily cross-reference, annotate, or repurpose (what is 
the license of each one?). I can, as above, refer to them by a URL, but 
then you need to talk to a central server to reference them. I can't 
summarize them a way that replaces the original which could be drilled 
down into, or collect other people's similar efforts and collaborate 
with them on that, mod emails like this one down as "tl;dr", :-) or 
alternatively tag them as insightful, all as part of a collaborative 
effort, and so on. Sure, I can kind of do all that with a lot of effort, 
especially if I set up (or find) a related Wiki somewhere central. :-) 
But I would hope we can do better.

At first, these could be tools on top of Thunderbird or FireFox or some 
website or whatever. But ideally, they should define some standard.

As I write this, I'm thinking, someone out there is probably even going 
to point out that there is a Thunderbird plugin somewhere for scarfing 
down Mailman archives and I could have done this specific task in a 
click and a drag of a URL from FireFox (assuming such a plugin exists 
and I did not think to look for it). A quick Google on "thunderbird 
plugin mailman" does not turn one up. But even if one exists, that 
misses the point about deeper issues in handling data in a semantic way. 
Stuff like the Nepomuk project worked towards, or which the Pointrel 
system I've worked on has been steps toward.

Personally, I haven't yet really made sense of NEPOMUK, but I have not 
spent much time with it or trying to install the whole thing. But at 
least it is a project that has had a lot of time put into it and is 
worthy as a concept of a lot more support:
http://nepomuk.semanticdesktop.org/xwiki/bin/view/Main/WebHome
"NEPOMUK brings together researchers, industrial software developers, 
and representative industrial users, to develop a comprehensive solution 
for extending the personal desktop into a collaboration environment 
which supports both the personal information management and the sharing 
and exchange across social and organizational relations."

These things all need more support, because:
   "Software is Hard"
   http://gamearchitect.net/Articles/SoftwareIsHard.html
"Scott Rosenberg coins this as Rosenberg's Law:  Software is easy to 
make, except when you want it to do something new.  The corollary is, 
The only software that's worth making is software that does something new."

Sure, build privacy stuff in to new tools and the FreedomBox, but our 
effectiveness to upgrade our society is IMHO more limited by the sorts 
of issues above leading to wheel spinning (not even mentioning wading 
through redundant content) than by lack of privacy and encryption and so 
on. Ultimately, giving someone encryption (or a gun for that matter) 
isn't really making the world a better place directly. These leaches on 
my time as far as problems in managing complex systems, leaches which 
FreedomBox is about to build onto the walls, may do way more damage to 
me and to our online communities than if anything I put on the computer 
(encrypted or not) is visible by the Feds or RIAA or whoever (and 
frankly, I assume everything is monitored by someone, anyway).

In fact, I try to see monitoring as a plus sometimes, as I suggest here: :-)
   http://www.pdfernhout.net/on-dealing-with-social-hurricanes.html
"Our biggest advantage is that no one takes us seriously. :-) And our 
second biggest advantage is that our communications are monitored, which 
provides a channel by which we can turn enemies into friends. :-) And 
our third biggest advantage is we have no assets, and so are not a 
profitable target and have nothing serious to fight over amongst 
ourselves. :-) Let's hope those advantages all hold true for a long 
time. :-)"

Although maybe I'm biased as I've helped build tools to help people make 
new content that is their own, so I think ultimately we can build a free 
society and leave behind the unfree content of those who don't want to 
share. I don't see that process has to be done in private. In fact, I 
feel it is better if it is done in public.

In any case, there are lots of ways to use technology to affirm freedom. 
I feel a social semantic desktop might make a big contribution towards 
that, and so integrating one into FreedomBox somehow would be a good 
idea. I think that would be more important than email. Unfortunately, 
semantic desktop stuff is still bleeding edge...

Other related content by me, the first being a FreedomBox-like proposal 
from ten years ago and also pre-shadowing the OLPC project:
   "[unrev-II] The DKR hardware I'd like to make..."
   http://www.dougengelbart.org/colloquium/forum/discussion/0754.html

And some code by me towards a semantic desktop (in Java, LGPL):
   "Pointrel Social Semantic Desktop" (Alpha, buggy, conceptual, 
passed-by by RDF and other stuff like maybe couchdb, etc.)
   http://sourceforge.net/projects/pointrel/

And ideas for revisions:
   "Towards a distributed Wiki"
   http://c2.com/cgi/wiki?PointrelWiki

And more ideas:
   "The need for better communication tools & a semantic web (was Re 
reprap-dev)"
   http://groups.google.com/group/openmanufacturing/msg/576771df555e729f

And more on generalizing the themes to collaborative sensemaking:
   "The need for FOSS intelligence tools for sensemaking etc."
   http://groups.google.com/group/openmanufacturing/msg/2846ca1b6bee64e1

Although, I'd really recommend looking at Apache's CouchDB as something 
that works right now and could be used in all sorts of ways in 
FreedomBox (and which I learned about from this mailing list). CouchDB 
at least gets a lot of the replication issues right (even if it still 
emphasizes on a walled-garden database sort of approach to data where 
data items are unique to databases but not the entire world).

Also in the interests of disclosure, I'll mention I worked on some 
sensemaking software with my wife related to DARPA/etc.'s TIA/Genoa-II 
project and Singapore's RAHS project, just so people don't think I'm 
hiding something. Sadly, so much "R&D" money has been poured into 
defense, usually in an ironic way, and there is so little for anything 
not labeled such. See:
http://www.pdfernhout.net/recognizing-irony-is-a-key-to-transcending-militarism.html

Although we got into those projects with my wife's hopes of helping 
decisions makers see things from a broader perspective (multiple 
perspectives, even), and that came after years and years of writing free 
software like our GPL'd Garden Simulator to help people with land learn 
how to grow their own food, but which did little for helping us put lots 
of food on the table for ourselves. Related:
   http://www.gardenwithinsight.com/

=== A final bug

Now I'm finally reading through the emails and seeing the ones labeled 
"Introductions" aren't introduction and no one changed the subject 
headings. But I can't go back and change that in a way that everyone can 
easily benefit from...

Anyway, for anyone in an age of "tl;dr" who made it to the end, thanks 
for reading. I'd much rather have put this on a wiki or whatever 
somewhere and just sent a short summary of it, but then it would not be 
distributed so there were backup copies everywhere.

I did not notice a license is regards to contributions to this email 
list. Please consider this document under a CC-BY-SA 3.0 license (same 
as Wikipedia) -- except for quoted material which is assumed to be under 
fair use in this context. But, that is another "bug" in email, no built 
in licensing information.

Anyway, that was a week ago, and I can't be bothered to go back and find 
half a days email to reimport, so I'm just going to live with a partial 
archive with some missing posts as I put a priority on other things..

So, in any case, I can hope for better in the future...

So, how important is email to FreedomBox again? :-)

All that said, I spend a lot of time with email and prefer it to lots of 
other things because it is a distributed system... Sure, it is better 
than a lot of other things.

--Paul Fernhout
http://www.pdfernhout.net/
====
The biggest challenge of the 21st century is the irony of technologies 
of abundance in the hands of those thinking in terms of scarcity.