Commit Graph

81 Commits

Author SHA1 Message Date
sebres
1eca571ea2 .zst, .br: avoid setting of size hint for unknown size by -si, since unpackSize will be supplied as -1 (UINT64_MAX) 2023-09-14 20:02:40 +02:00
sebres
8a5e1c82db make compression of zstd archive type more similar to Zstandard CLI (store dictID and checksum by default in zstd type, no effect for 7z type) 2023-09-14 18:23:11 +02:00
sebres
5697b3dece zstd compression - set source size as hint if it is known e. g. by file compression (slightly better performance and/or compression ratio);
although the feature still calling as "experimental", but zstd uses this in its own client since v.1.4 IIRC and the only known drawback would be significant regress of compression ration if guess considerably underestimates, but it does no matter in case of known file size.
2023-09-14 18:23:10 +02:00
sebres
2b48366507 if single-threaded brotli, retain artificial number set in BrotliHandler (0) - always prefer it for .br format, otherwise -mmt1 causes that it'd use brotli-mt and the stream is incompatible; -mmt=n with n >= 2 forces multi-threaded compression/decompression (therefore incompatible with standard brotli) 2023-09-07 20:29:35 +02:00
sebres
eeae03eaa1 Add options to brotli and implement clean brotli .br support
- allow to specify brotli window size
  - parameter -m0=brotli:long=n, BROTLI_MAX_WINDOW_BITS (24) used by default in brotli-mt, smaller == faster
  - note that :long can be set up to BROTLI_LARGE_MAX_WINDOW_BITS (30), whereas :wlog can be set up to BROTLI_MAX_WINDOW_BITS (24) only...
  - todo: check whether set of BROTLI_PARAM_LARGE_WINDOW to BROTLI_TRUE is needed if (lgwin > BROTLI_MAX_WINDOW_BITS)

- implementation of single-threaded brotli compression / decompression for .br data

Signed-off-by: Sergey G. Brester <info@sebres.de>
Reviewed-by: Tino Reichardt <milky-7zip@mcmilk.de>
2023-09-06 17:56:30 +02:00
sebres
ebe8399e0f resolve deprecated ZSTD_resetDStream with its equivalent ZSTD_DCtx_reset(_ctx, ZSTD_reset_session_only); 2023-02-15 15:34:55 +01:00
Tino Reichardt
f9e0730191 Update to 7-Zip Version 22.01
See: https://sourceforge.net/p/sevenzip/discussion/45797/thread/c43cbc5f18/
2022-08-07 10:06:40 +02:00
Tino Reichardt
57558682a8 Update to 7-Zip Version 22.00
See: https://sourceforge.net/p/sevenzip/discussion/45797/thread/9c2d9061ce/
2022-08-07 10:06:40 +02:00
Tino Reichardt
6a4fe97fc3 Update to 7-Zip Version 21.07
See: https://sourceforge.net/p/sevenzip/discussion/45797/thread/0a7b47970b/
2022-08-07 10:06:40 +02:00
Tino Reichardt
09497b7ba0 Update to 7-Zip Version 21.04
- first test... no release!!!
2021-11-06 22:17:34 +01:00
Tino Reichardt
0f6bcfd2ed Remove unused debug code in Compress/Zstd*.cpp 2021-10-05 23:17:27 +02:00
Tino Reichardt
df06f31a42 Update to 7-Zip Version 21.03 2021-08-26 04:33:47 +02:00
Tino Reichardt
48fa49f76c Update to 7-Zip Version 21.02 2021-05-13 19:16:55 +02:00
jinfeihan57@gmail.com
03cc9ede2b fix Bug #141 2020-12-12 23:25:26 +01:00
jinfeihan57@gmail.com
d2c9935159 make progress bar smoother 2020-11-17 18:31:12 +08:00
Tino Reichardt
9378903648 Fix zstd decompression, files with large window settings did not work 2020-06-02 09:34:09 +02:00
Tino Reichardt
2890284f67 Fix #130 - incorrect progress number with zstd 2020-06-02 08:50:49 +02:00
Tino Reichardt
9c3c277ad7 Normalize all the line endings 2020-05-31 13:08:03 +02:00
Tino Reichardt
256e46435f Fix #131 2020-05-29 15:06:05 +02:00
Tino Reichardt
9884fb96fa If zstd fails, give a more specific failure massage, fix #109 2020-04-14 10:18:58 +02:00
Tino Reichardt
d2845a69d0 Remove _hMutex from ZstdEncoder.cpp 2020-04-14 07:05:38 +02:00
Tino Reichardt
16b902a89f LZ4 compression defaults to one thread 2020-04-10 14:44:24 +02:00
Tino Reichardt
d17138af60 Fix #103 and comments for level and version numbers 2020-04-10 14:44:24 +02:00
Tino Reichardt
5e3efe410d Compiling for IA64 and WinXP does not work, cause FLZMA2 doesn't build then :/ 2019-11-03 16:28:05 +01:00
Tino Reichardt
621c9fc012 Disable new C++11 expression in Lzma2Encoder.h 2019-11-03 11:33:20 +01:00
Tino Reichardt
2f4bcd7f4c Fix issue #88, thanks @xzn 2019-06-24 19:10:39 +02:00
Tino Reichardt
7932a50599 Merge pull request #85 from conor42/master
Pass blockSize to fast-lzma2 and limit it to 256 Mb
2019-05-04 22:45:59 +02:00
conor42
0cc8701c38 Pass blockSize to fast-lzma2 and limit it to 256 Mb 2019-05-02 12:51:03 +10:00
Tino Reichardt
4b0de7ff78 Disable C4127: Conditional Expression is Constant for Lzma2Encoder.cpp 2019-03-29 20:56:57 +01:00
conor42
ed9073e9fe Add error check 2019-03-29 12:13:14 +10:00
conor42
80ae5388ad Fix x86 warning and improve progress updates 2019-03-29 00:14:05 +10:00
conor42
4b0103cebd Update fast-lzma2 2019-03-22 20:14:34 +10:00
conor42
26cd887a90 Merge branch 'pr/1' 2019-03-18 00:21:14 +10:00
conor42
d85962e654 Update to Fast LZMA2 1.0.0 2019-03-18 00:05:50 +10:00
vinnyrom
cab7bc1a72 Fix null dereference of 'progress' when updating an archive. The callstack of the crash is:
zstd-x64.dll!NCompress::NZSTD::CDecoder::CodeSpec(ISequentialInStream * inStream=0x000000001e6a2190, ISequentialOutStream * outStream=0x000000001e6b9460, ICompressProgressInfo * progress=0x0000000000000000) Line 130 C++
  zstd-x64.dll!NCompress::NZSTD::CDecoder::Code(ISequentialInStream * inStream=0x000000001e6a2190, ISequentialOutStream * outStream=0x000000001e6b9460, const unsigned __int64 * __formal=0x000000001e8a2d00, const unsigned __int64 * outSize=0x000000001e5eb3b0, ICompressProgressInfo * progress=0x0000000000000000) Line 177 C++
  7z.dll!NCoderMixer2::CCoderMT::Code(ICompressProgressInfo * progress=0x0000000000000000) Line 903 C++
  7z.dll!NCoderMixer2::CMixerMT::Code(ISequentialInStream * const * inStreams=0x000000001e8a2d20, ISequentialOutStream * const * outStreams=0x00000000219bf910, ICompressProgressInfo * progress=0x0000000000000000, bool & dataAfterEnd_Error=false) Line 1083 C++
  7z.dll!NArchive::N7z::CDecoder::Decode(const CExternalCodecs * __externalCodecs=0x000000001e599aa0, IInStream * inStream=0x000000001e8c6300, unsigned __int64 startPos=32, const NArchive::N7z::CFolders & folders={...}, unsigned int folderIndex=0, const unsigned __int64 * unpackSize=0x0000000000000000, ISequentialOutStream * outStream=0x000000001e6b9460, ICompressProgressInfo * compressProgress=0x0000000000000000, ISequentialInStream * * inStreamMainRes=0x0000000000000000, bool & dataAfterEnd_Error=false, ICryptoGetTextPassword * getTextPassword=0x000000001e6882b0, bool & isEncrypted=true, bool & passwordIsDefined=true, UString & password={...}, bool mtMode=false, unsigned int numThreads=1, unsigned __int64 memUsage=0) Line 555 C++
  7z.dll!NArchive::N7z::CThreadDecoder::Execute() Line 1493 C++
  7z.dll!CoderThread(void * p=0x000000001f1cbe90) Line 16 C++

Other 7z threads are:

  7z.dll!Handle_WaitObject(void * h=0x0000000000000868) Line 32 C
  7z.dll!NWindows::NSynchronization::CBaseEvent::Lock() Line 47 C++
  7z.dll!CStreamBinder::Read(void * data=0x000000001e9b53ea, unsigned int size=115206, unsigned int * processedSize=0x000000001f1cb310) Line 95 C++
  7z.dll!CBinderInStream::Read(void * data=0x000000001e9b53ea, unsigned int size=115206, unsigned int * processedSize=0x000000001f1cb310) Line 22 C++
  7z.dll!COffsetOutStream::Write(const void * data=0x000000001e9b53ea, unsigned int size=115206, unsigned int * processedSize=0x000000001f1cb310) Line 19 C++
  7z.dll!CSequentialInStreamSizeCount2::Read(void * data=0x000000001e9b53ea, unsigned int size=115206, unsigned int * processedSize=0x000000001f1cb350) Line 10 C++
  zstd-x64.dll!ReadStream(ISequentialInStream * stream=0x000000001e7b4010, void * data=0x000000001e9b53ea, unsigned __int64 * processedSize=0x000000001f1cb3b8) Line 17 C++
  zstd-x64.dll!NCompress::NZSTD::CEncoder::Code(ISequentialInStream * inStream=0x000000001e7b4010, ISequentialOutStream * outStream=0x000000001e8e0dc0, const unsigned __int64 * __formal=0x0000000000000000, const unsigned __int64 * __formal=0x0000000000000000, ICompressProgressInfo * progress=0x000000001b6df8d0) Line 329 C++
  7z.dll!NCoderMixer2::CCoderMT::Code(ICompressProgressInfo * progress=0x000000001b6df8d0) Line 903 C++
  7z.dll!NCoderMixer2::CMixerMT::Code(ISequentialInStream * const * inStreams=0x000000001f1cb750, ISequentialOutStream * const * outStreams=0x000000001e599bc0, ICompressProgressInfo * progress=0x000000001b6df8d0, bool & dataAfterEnd_Error=false) Line 1083 C++
  7z.dll!NArchive::N7z::CEncoder::Encode(const CExternalCodecs * __externalCodecs=0x000000001e599aa0, ISequentialInStream * inStream=0x000000001e7b3f10, const unsigned __int64 * inSizeForReduce=0x000000001f1cb960, NArchive::N7z::CFolder & folderItem={...}, CRecordVector<unsigned __int64> & coderUnpackSizes={...}, unsigned __int64 & unpackSize=17468, ISequentialOutStream * outStream=0x000000001e8bcdd0, CRecordVector<unsigned __int64> & packSizes={...}, ICompressProgressInfo * compressProgress=0x000000001b6df8d0) Line 454 C++
  7z.dll!NArchive::N7z::Update(const CExternalCodecs * __externalCodecs=0x000000001e599aa0, IInStream * inStream=0x000000001e8c6300, const NArchive::N7z::CDbEx * db=0x000000001e5998a8, const CObjectVector<NArchive::N7z::CUpdateItem> & updateItems={...}, NArchive::N7z::COutArchive & archive={...}, NArchive::N7z::CArchiveDatabaseOut & newDatabase={...}, ISequentialOutStream * seqOutStream=0x000000001e8bcdd0, IArchiveUpdateCallback * updateCallback=0x000000001f1cc980, const NArchive::N7z::CUpdateOptions & options={...}, ICryptoGetTextPassword * getDecoderPassword=0x000000001f1cc958) Line 2178 C++
  7z.dll!NArchive::N7z::CHandler::UpdateItems(ISequentialOutStream * outStream=0x000000001e8bcdd0, unsigned int numItems=10, IArchiveUpdateCallback * updateCallback=0x000000001f1cc980) Line 658 C++

And:

  7z.dll!Handle_WaitObject(void * h=0x0000000000000880) Line 32 C
  7z.dll!NWindows::NSynchronization::CBaseEvent::Lock() Line 47 C++
  7z.dll!CStreamBinder::Read(void * data=0x0000000020650000, unsigned int size=1048576, unsigned int * processedSize=0x000000002084f6d0) Line 95 C++
  7z.dll!CBinderInStream::Read(void * data=0x0000000020650000, unsigned int size=1048576, unsigned int * processedSize=0x000000002084f6d0) Line 22 C++
  7z.dll!ReadStream(ISequentialInStream * stream=0x000000001e8e0de0, void * data=0x0000000020650000, unsigned __int64 * processedSize=0x000000002084f758) Line 17 C++
  7z.dll!CFilterCoder::Code(ISequentialInStream * inStream=0x000000001e8e0de0, ISequentialOutStream * outStream=0x000000001e55da30, const unsigned __int64 * __formal=0x0000000000000000, const unsigned __int64 * outSize=0x0000000000000000, ICompressProgressInfo * progress=0x0000000000000000) Line 83 C++
  7z.dll!NCoderMixer2::CCoderMT::Code(ICompressProgressInfo * progress=0x0000000000000000) Line 903 C++
  7z.dll!NCoderMixer2::CCoderMT::Execute() Line 867 C++
  7z.dll!CoderThread(void * p=0x000000001e5eb510) Line 16 C++
2019-01-15 16:42:11 -08:00
Tino Reichardt
a488536124 Update to 7-Zip Version 18.06 2018-12-30 13:30:20 +01:00
Tino Reichardt
7595ca1994 Update zstd to version 1.3.8 2018-12-28 15:34:32 +01:00
Tino Reichardt
1d1e92a9fb Fix decompression of zstd again
- zIn.size was not correctly set before first decompression loop
2018-12-05 22:28:58 +01:00
Tino Reichardt
28bc209464 Revert the update of the 7-Zip property sizes
- change the 7-Zip property size for LZ4, LZ5 and Zstandard back to 5
- the code understands 3 and 5 bytes, but we default to 5 ...
2018-12-04 21:11:23 +01:00
Tino Reichardt
bb06376057 Fix decompression of skippable zstd frames 2018-12-04 21:07:12 +01:00
Tino Reichardt
f98edef556 Add zstd fast levels and update 7z property sizes
- add the "fast compression levels" of zstd via "fast" option (fast=1..64)
- change the 7-Zip property sizes of LZ4, LZ5 and Zstandard to 3
- 3 and 5 byte header are valid now (default is 3)
- update the Methods-Extern.md file, to reflect the property changes
2018-11-25 21:21:04 +01:00
Tino Reichardt
4728ce24ef Autoadjust ldm option on large windowLog values 2018-11-18 22:17:23 +01:00
Tino Reichardt
714d514451 Make zstd:long behave like in zstd cli
- adjust version information to current date
- maybe I release this as the next version
2018-11-17 11:08:59 +01:00
Tino Reichardt
a24bf9aca1 Add zstd advanced compression options 2018-11-16 22:34:41 +01:00
conor42
f531a44f1c Update readme; Add IA64 dll build 2018-11-09 20:30:35 +10:00
conor42
0cbdf835f0 Merge remote-tracking branch 'upstream/master' 2018-11-07 14:34:48 +10:00
conor42
296c91d212 Add Fast LZMA2 codec dll build; Fix 32-bit warnings 2018-11-07 14:34:04 +10:00
Tino Reichardt
eff4d316fa Merge pull request #58 from conor42/master
Add Fast LZMA2 codec
2018-11-05 17:29:02 +01:00
conor42
ab10047253 Add Fast LZMA2 codec 2018-11-05 21:22:10 +10:00
Tino Reichardt
1a09f6fc0d Fix zstd decompression
- todo: make zstd ldm compression optional
2018-11-03 10:03:54 +01:00