Bug#631295: gvfs-backends: gvfsd-gphoto2 handles large image/video files very badly

[Jack Whitham]

Package: gvfs-backends
Version: 1.6.4-3
Severity: important
Tags: upstream


This bug can be reproduced by 
1. recording a large video file (1000Mb+) using a digital camera,
2. attaching that camera to the PC,
3. open the camera folder that appears on the desktop, then 
find the video file using the File Browser (nautilus),
4. copy the file onto the desktop by drag and drop.


The symptoms are as follows:
1. The entire file is copied into memory before being written to
$HOME/Desktop, potentially causing a swap storm or even crashing the PC.
Using "top" you can observe the size of the gvfsd-gphoto2 process
increasing during the copy operation.
2. The file is apparently downloaded from the camera more than once.
3. The copy progress bar appears, but sits at 0% for some time.


I have investigated the first symptom and traced it to code in
gvfsbackendgphoto2.c within the gvfs-1.6.4 source package. 
The function "do_open_for_read_real" is responsible for copying each
file from the camera, which is done using code in libgphoto2.

The "gp_file_new" function is used to allocate a CameraFile object
to temporarily store the data. Unfortunately, "gp_file_new" stores the 
data entirely in memory: fine for images of a few megabytes, but entirely
unsuitable for gigabyte-sized video files. 

An examination of the command line tool "gphoto2", which also uses 
libgphoto2, suggests that "gp_file_new_from_fd" would be a more 
appropriate choice. When "gp_file_new_from_fd" is used instead,
the memory usage of gvfsd-gphoto2 remains reasonable.

However, this fix just reveals the second problem, namely that each 
file will be downloaded more than once -- before the progress
bar even begins to move. Again, this is not noticeable for small 
files, but it is a major annoyance for large copies. Furthermore,
large copies may never complete, as the operation appears to time out.

If the problem were limited to the first symptom, namely the apparent 
misuse of "gp_file_new", then I could easily suggest a patch. But it 
appears that there is a deeper design problem, namely that:
1. A file is apparently downloaded from the camera more than once.
2. The copy progress bar appears, but sits at 0% for some time, whilst
those copies complete.


Further information:
1. The latest upstream version of gvfsbackendgphoto2.c also uses 
gp_file_new. 
2. To my knowledge the bug is not open within Debian, but it has 
apparently been reported in Ubuntu (note same symptoms):
https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/782933
https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/627908




-- System Information:
Debian Release: 6.0.1
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/1 CPU core)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages gvfs-backends depends on:
ii  gvfs              1.6.4-3                userspace virtual filesystem - ser
ii  libarchive1       2.8.4-1                Single library to read/write tar, 
ii  libavahi-client3  0.6.27-2+squeeze1      Avahi client library
ii  libavahi-common3  0.6.27-2+squeeze1      Avahi common library
ii  libavahi-glib1    0.6.27-2+squeeze1      Avahi glib integration library
ii  libbluetooth3     4.66-3                 Library to use the BlueZ Linux Blu
ii  libc6             2.11.2-10              Embedded GNU C Library: Shared lib
ii  libcdio-cdda0     0.81-4                 library to read and control digita
ii  libcdio-paranoia0 0.81-4                 library to read digital audio CDs 
ii  libcdio10         0.81-4                 library to read and control CD-ROM
ii  libdbus-1-3       1.2.24-4               simple interprocess messaging syst
ii  libdbus-glib-1-2  0.88-2.1               simple interprocess messaging syst
ii  libexpat1         2.0.1-7                XML parsing C library - runtime li
ii  libgconf2-4       2.28.1-6               GNOME configuration database syste
ii  libglib2.0-0      2.24.2-1               The GLib library of C routines
ii  libgphoto2-2      2.4.6-3                gphoto2 digital camera library
ii  libgphoto2-port0  2.4.6-3                gphoto2 digital camera port librar
ii  libgudev-1.0-0    164-3                  GObject-based wrapper library for 
ii  libimobiledevice1 1.0.2-1                Library for communicating with the
ii  libplist1         1.3-2                  Library for handling Apple binary 
ii  libsmbclient      2:3.5.6~dfsg-3squeeze2 shared library for communication w
ii  libsoup-gnome2.4- 2.30.2-1               an HTTP library implementation in 
ii  libsoup2.4-1      2.30.2-1               an HTTP library implementation in 
ii  libxml2           2.7.8.dfsg-2+squeeze1  GNOME XML library

Versions of packages gvfs-backends recommends:
ii  gnome-keyring                 2.30.3-5   GNOME keyring services (daemon and

Versions of packages gvfs-backends suggests:
ii  obex-data-server              0.4.5-1+b1 D-Bus service for OBEX client and 

-- no debconf information