[med-svn] [htslib] branch master updated (5a0ee03 -> 26229a3)
Charles Plessy
plessy at moszumanska.debian.org
Mon May 25 02:38:49 UTC 2015
This is an automated email from the git hooks/post-receive script.
plessy pushed a change to branch master
in repository htslib.
from 5a0ee03 Release 1.1: various minor bug fixes
adds f5aeebb bgzf_open(..,"g") for compressed gzip output, as opposed to BGZF
adds f2080d5 Merge version number bump from master
adds 5b98adc Added regidx API which should replace bcf_sr_regions at some point
adds f4b4b61 Clean after regidx init failure; fixed build dependencies
adds d27ae83 Fix in bcf regions which in some situations would skip first
adds d2cb7ba regidx: simple file format autodetection
adds c116e11 add new test-regidx binary to .gitignore
adds db774cc Merge pull request #136 from mcshane/feature/ignore_test-regidx
adds 14a4a81 Return type of bgzf_getc is int, not char
adds 56d50b2 regidx: use size_t, not ssize_t
new 5ebf9b2 Fix regidx.o dependencies and htslib.mk; alphabetise
new a0e35e0 Formatting fixes for man page
new effc6fd Merge man page formatting fixes
new 912a7d0 Support for Type=Character, in htslib same as Type=String
new 34f8089 VCF header editing speedup for large number of ref sequences
new 9d01cd6 New bcf_copy API
new 061cd10 Minor speed increases to cram_byte_array_stop_decode_init and GET_BIT_MSB macros.
new 6454bd4 Added a range coder (order 0 and 1) plus support for LZMA if compiled in (no option for this at present).
new 94cf60a Tweaks for specifying version number. Now accepting version 3.0, and also correctly setting the globals.
new 3d3f400 Updated the version checks to lop off major/minor components rather than using an exact == comparison.
new 37f42cd Fixed BETA codec so that it honours beta offset value for zero length codes.
new 1de851b Bug fix to the external decoders. If there is an attempt to decode 0 bytes then it no longer matters if the block does not exist. (This comes about when faced with silly CIGAR strings like "0S".)
new 0d39459 Added support for more codecs, as part of refactoring how data is pushed. These were in the V2.1 spec, but not used.
new 672f079 Major refactoring of the way CRAM handles external blocks. We now prefer to output as many things as possible to their own specific external block instead of utilising the CORE block more often. This has the impact that it is much easier to do a partial decode.
new 100f2d0 Removal of a couple memory leaks added in the restructing of the last patch.
new b73b527 Bug fix of use of TAG_ID macro and k vs key in hash for auxiliary headers.
new 5a840da Modified the thread pool to use as few threads as possible. The intention is that when given, say, 16 threads but being I/O bound such that 12 cores is enough to keep up with the I/O then we have 12 threads kept busy and 4 totally idle.
new 6929b9c Bug fixed scramble -x as it was not correctly setting the RI data series for some tests.
new ef7eeed A mishmash of changes for CRAM v3.0 (not yet the default output format). Sorry it's munged together.
new a254285 Minor gcc warnings fixups.
new 2e29861 Added support for compressed SAM headers.
new c7f8fee Final part of Staden io_lib's commit r3686.
new 6d01daa Initialise refs_t ->ref_id in refs_load_fai(). This has no bearing on Samtools/htslib, but this function is used within Gap5 and calling it in this order gave arise to crashes unless this initialisation code is here, so it is a good belt and braces approach.
new 22718e5 Added CRC32 to the blocks and containers.
new 3ca319f Fixed the file format detection code so it handles CRAM v3.0 magic number as a valid CRAM file.
new 10958ff Changed the hts_open code to support "C" as a format mode as well as "c". Both are CRAM, but "C" is version 3.0. This is a temporary hack while CRAM V3.0 is still undergoing work, but we will need a more formal way of indicating output version numbers in the future. Equally so the CRAM code itself for specifying versions also needs improvements, to avoid global variables. Test_view has a -3 option to use "C" instead of "c".
new bf2d07c Fixed bam_construct_seq to cope with qual being NULL.
new efa6537 Removed the previous "c" vs "C" hack and added a proper option parser to test_view. We still need to decide where this API belongs properly, but test_view is a good point to test it.
new 1965be2 Fixed generation of MD and NM tags in cram_decode_seq() when using a non-reference encoding. These cannot be stored or regenerated in such cases.
new b56f540 Added a special case for setting the reference. When converting from SAM to CRAM we tend to think of the reference as an input property; if it is not in the SAM headers (M5 & UR @SQ tags) then we want to specify it, as an input property.
new ad3c4c5 Cope with cram_compress_slice failing. We were unwinding the stack returning -1 each time, but then called cram_close which attempting to flush any remaining data (dying in the process). We now free the current on-going container upon an error, to avoid this flush later.
new 1ef04ee Removed the need for -DSAMTOOLS when compiling CRAM.
new b5ecdf7 (Commented out): ugly auto-configuration of HAVE_LIBBZ2 and HAVE_LIBLZMA definitions.
new a223cae Added multi-threading support for reading and writing CRAM.
new 4835f53 CRAM_OPT_VERSION now works (and only works) on an open file descriptor, freshly opened before the call to sam_hdr_write.
new ac13591 Added SAM_RGAUX as another column identifier to allow for tools that need to use RG tag without other tags to perform optimally.
new 2f10ff9 Removed defunct comment.
new 0cdb7b1 Minor tidyups to prevent some clang warnings.
new eecc982 Fixes for handling range requests while also multi-threading.
new d6cdff6 Fix for FSECONDARY reads; do not link into PNEXT/RNEXT.
new 9eb4bed Removed various small memory leaks.
new 432a224 Split off the rans_byte.h portion of rANS_static.c back into its own file, for reasons of copyright clarity.
new b5d11c3 Added copyright notice.
new 43f2d11 Remove -DSAMTOOLS vestiges
new 24c8699 Fix cram/rANS_* dependencies
new 233e159 Merge CRAM v3 updates (PR #132)
new abd1efb Add htsFormat and format-detection API functions
new 529ca88 Use htsFormat to replace htsFile's is_foo flags
new 41ab01b Add htsfile utility
new bf909d6 Removed old hts_file_type() API and replaced it with the new htsFormat;
new 7c42dcc Clean up, is_compressed and is_cram no longer necessary
new eda497b Do not use 0x80 & co for missing alleles
new 3e24dfd Do not load remote index if already exists locally
new 5dec96b Amended the compression level checking code in cram_dopen() to follow the same detection logic used in bgzf.c. Previously changing the compression level did not work for CRAM.
new b0df3d1 Fix in bgzf's gzip reading plus added a test for this
new 9c510fb bcf_translate: Be aware of gaps in BCF headers
new b28efa4 In a bid to keep the DEBUG_printf line potentially printing up the value of 'i', while also avoiding the complaints about 'i' being set but unused, I changed the code to also handle potentially wide characters. (I haven't tested it works actually with wide characters, but it'll be better than before and the main goal was the silence the annoying warning!)
new a98d88f Fix a bug where hts_set_fai_filename() didn't pass this through to CRAM. It appears this bug has been long standing, since https://github.com/samtools/htslib/commit/2402fc00fe1f2360cd9056173045f65bc0b683dc, but was not detected due to the tests finding the reference via a UR: @SQ tag instead.
new 9e844e0 Fix bug spotted by valgrind when running "./test_view -D ce#unmap.tmp.cram".
new 7a13d83 Fixed an error found via valgrind of test_view -D c1#pad1.tmp.cram.
new dd08ee0 Reverting 9e844e0be26875d29b6e2c853c2b4d02c4f1faa4 and rewriting in a better manner.
new 244dde8 tbx: Detect faulty tbi files
new b7f74f4 Merge origin/develop, replacing htsFile.type by .format
new 845c515 Parse SAM aux 'i' values > 2^31 correctly
new 07c94ec Sped up cram_index_load some 200 fold by replacing sscanf with our own number decoding.
new b96a302 Fixed an incorrect EOF-style case when seeking multiple times.
new 4d13ff0 Fix of memmove bug in bcf_remove_filter()
new 4aa494f Buffer overflow error in synced_bcf_reader.c
new c4043c5 Merge pull request #142 from reinders/patch-1
new bd6f52a Fix compilation when ALLOW_UAC is not defined
new 7838da8 bcf_sr_add_reader(): do not increment nreaders or perform any reallocs upon file open errors
new 54618dd Merge pull request #144 from broadinstitute/broad_bcf_sr_add_reader_segfault_fix
new ee7343e bcf_sr_get_header() macro for accessing the synced readers' headers
new 29305dd Add IDX to hdr tags of different type, fixes issue https://github.com/samtools/bcftools/issues/141
new 83a4e30 Amended/removed comments about scram_* API.
new 089c900 Merge short read() bug fix from upstream
new 3c4f33a Fix ks_getuntil2() extra empty record at EOF bug
new d8c03cf bcf_hdr_subset: Return NULL on duplicate sample names
new 8916744 Sanity check to detect broken GT fields in bcf_calc_ac()
new 9a88137 Add seq_nt16_int[], equivalent to the old API's bam_nt16_nt4_table[]
new bf7e0ec bcf_*hrec* functions: check for existing/multiple IDX keys
new 4770a41 Fixed the generation of read names to use record_counter properly, counting from 1, rather than slice:record-in-slice.
new 876bfe4 Replaced sprintf with a home-brew append_uint64 function. This is a sizeable speed increase to the read name auto-generation code.
new 89bfcc4 Fix comment in sam.h
new 32a43b2 vcf: skip empty INFO tags ";;". (Error might be more appropriate?)
new 8d921a5 Account for read buffering in hseek(SEEK_CUR)
new 57462fb Minor improvement to binary searching in CRAM indices.
new 9da961a bcf_hdr_combine: Complain when trying to merge different types
new 7a0fdf4 Sanity-check tid in hts_itr_query()
new 10ed34e bcf_calc_ac: Check for incorrect AC/AN counts
new cef706f vcf_parse_format: Throw an error on extra FORMAT fields
new dfd6773 vcf: Propagate hdr_add_sample() error
new 5e76b1c Fix alleles trimming with format Number=R/A/G tags
new 948a68c Removed spurious messages about missing EOF blocks in CRAM when dealing with older versions of the file format. It worries users to see messages about lack of an EOF block (although technically true) when reading v2.0 or earlier CRAM files.
new ce1a547 Parse regions without begin/end as 1..MAX_INT rather than 1..2^29
new e5a964e Update khash.h from upstream sources
new 20238f3 Fix various simple memory leaks (cf #138)
new fe88482 Added in the compressed length field to the rANS codec header.
new ef59ef2 Deobfuscate memory allocations etc via sizeof()
new 6ee481f vcf headers: Allow contig lines without length attribute. Resolves #155
new 547a349 tabix: Remove bcf and bam from presets,
new 21fbc8b vcf: Abort on duplicate sample names, resolves #184
new ed3efe9 New -R/-T options to tabix.
new f3e1602 Fix a typo, VCF contig length should be stored.
new dcffda5 Two changes to cram_encode_container() related to efficient encoding of name-sorted data.
new ca6f60e Fixed a memory leak when destroying a BYTE_ARRAY_LEN encoder.
new f7caefc Fixed a small memory leak where we didn't deallocate a cram_block that we had created but later culled due to containing zero bytes.
new cba1bf0 Fixed memory leak when trying to O1 compress a block <= 4 bytes long.
new 8ad2912 Make the multi_seq parameter default to auto. This is the default in Scramble, but oddly not the default in the code (so scramble always reset it from 0 to -1).
new 2b31b7d More ref and memory management fixes.
new ab23898 Reverted the cram_encode_container change to call cram_ref_incr and added a ref incr in cram_get_ref instead. This better fixes the issue of sharing references between containers, fixing decoding as well as encoding.
new 11a33a6 Merge pull request #160 from jkbonfield/fix_unsorted_cram
new 5f7a4ea Fixed a bogus warning about using 'cp' before initialised. (Bogus as it's promptly reassigned again, so tidied up the code.)
new 9b1cb94 Avoid aux.* filenames, which are invalid on Windows
new 3768707 Update faidx.h
new 3ac7d00 Add configure.ac script
new 2ac7a82 Add notes for building from a Git repository
new 0ccc935 Add iRODS hFILE backend
new 8bc776d Add MIT/Expat license boilerplate
new 83f1dbc Merge iRODS hFILE backend
new 25e8fac Minor change to allow REQUIRED_FIELDS option to be specified in hex or octal.
new dd70964 Overhauled the cram_dependent_data_series function and associated code.
new 7bd8c08 Merge pull request #161 from jkbonfield/cram_dependent_data_series
new ef3bd19 Document ./configure --with-irods
new 91a471d Fixed unnecessary FAI building.
new 97a7933 Bug fix to refs_from_header().
new 0ad9965 Add hisremote(), and convert faidx.c from knet to hFILE
new cf4811a Set resource when writing to iRODS
new 519e0e7 Add htsfile(1) man page
new b2cfe4e Remove tabix -i, which duplicates htsfile functionality
new 61a79f1 Better index bugfix
new 2244263 Detect file format versions in hts_detect_format()
new 0ec1bb1 Temporarily avoid rcDataObjFsync() [workaround]
new e13b690 Fixes imported from Staden io_lib revisions 3792/3795.
new 8c80202 Additional comments.
new 3ec78c1 Improved the CRAM stats array usage.
new 92a0129 Merge CRAM TLEN updates (PR #165)
new c360ce4 Enforce the use of a local cache (use home dir if not defined) whenever we automatically fall back to using the EBI reference sequence server.
new 77cdbec Use TMPDIR and if not set TEMP (common on Windows) environment variables as the location of temporary files, in preference to a hard coded /tmp.
new 0c74c75 Stylistic code change: !*ptr vs *ptr=='\0'.
new 32b534f Added support for XDG_CACHE_HOME.
new 1493ea0 Avoid hiding under .cache in temp directories
new 6c6f02e Merge default local reference cache (PR #166)
new af9768d Rationalise include guard macro name
new 9ecdaae Move remainder of config.h to bgzf.c and remove it
new 38d93e2 Reinstate faidx_fetch_nseq() alongside faidx_nseq()
new 38b5374 Formatting fix for HTML man page
new bff5efb Release 1.2: various bug fixes, htsfile utility, CRAM improvements, etc
new b0742b0 Merge version number bump and NEWS file from master
new 7ebc5ae Reinstate deprecated hts_file_type() and FT_*
new 26229a3 Release 1.2.1: patch release over 1.2, reinstating hts_file_type()
The 151 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "adds" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 9 +
INSTALL | 88 +-
Makefile | 91 +-
NEWS | 50 +
README.md | 16 +
bgzf.c | 58 +-
bgzip.c | 2 +-
config.h | 3 -
config.mk.in | 72 ++
configure.ac | 93 ++
cram/cram.h | 9 +-
cram/cram_codecs.c | 230 ++--
cram/cram_codecs.h | 22 +-
cram/cram_decode.c | 1339 ++++++++++++++++------
cram/cram_encode.c | 1994 ++++++++++++++++++++-------------
cram/cram_index.c | 82 +-
cram/cram_io.c | 1382 ++++++++++++++++-------
cram/cram_io.h | 98 +-
cram/cram_samtools.c | 5 +-
cram/cram_stats.c | 123 +-
cram/cram_structs.h | 329 +++---
cram/os.h | 2 +
cram/rANS_byte.h | 336 ++++++
cram/rANS_static.c | 841 ++++++++++++++
cram/rANS_static.h | 44 +
cram/sam_header.c | 8 +-
cram/sam_header.h | 8 -
cram/thread_pool.c | 178 ++-
cram/thread_pool.h | 15 +-
cram/vlen.c | 2 +-
faidx.c | 46 +-
hfile.c | 23 +-
hfile_internal.h | 3 +-
hfile_irods.c | 243 ++++
hts.c | 531 +++++++--
htsfile.1 | 71 ++
htsfile.c | 168 +++
htslib.mk | 8 +-
htslib/bgzf.h | 7 +-
htslib/faidx.h | 2 +-
htslib/hfile.h | 10 +-
htslib/hts.h | 146 ++-
htslib/khash.h | 8 +-
htslib/khash_str2int.h | 9 +
htslib/kseq.h | 10 +-
htslib/regidx.h | 147 +++
htslib/sam.h | 2 +-
htslib/synced_bcf_reader.h | 15 +-
htslib/vcf.h | 15 +-
htslib_vars.mk | 1 +
knetfile.c | 11 +-
regidx.c | 338 ++++++
sam.c | 134 ++-
synced_bcf_reader.c | 88 +-
tabix.1 | 121 +-
tabix.c | 268 +++--
tbx.c | 5 +
test/{aux#aux.sam => auxf#values.sam} | 0
test/{aux.fa => auxf.fa} | 0
test/{aux.fa.fai => auxf.fa.fai} | 0
test/hfile.c | 4 +-
test/sam.c | 52 +-
test/test-regidx.c | 116 ++
test/test-vcf-api.c | 57 +-
test/test_view.c | 92 +-
vcf.c | 207 ++--
vcfutils.c | 63 +-
67 files changed, 8134 insertions(+), 2416 deletions(-)
create mode 100644 NEWS
delete mode 100644 config.h
create mode 100644 config.mk.in
create mode 100644 configure.ac
create mode 100644 cram/rANS_byte.h
create mode 100644 cram/rANS_static.c
create mode 100644 cram/rANS_static.h
create mode 100644 hfile_irods.c
create mode 100644 htsfile.1
create mode 100644 htsfile.c
create mode 100644 htslib/regidx.h
create mode 100644 regidx.c
rename test/{aux#aux.sam => auxf#values.sam} (100%)
rename test/{aux.fa => auxf.fa} (100%)
rename test/{aux.fa.fai => auxf.fa.fai} (100%)
create mode 100644 test/test-regidx.c
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/debian-med/htslib.git
More information about the debian-med-commit
mailing list