# data[].attributes.body
Is an algorithim

# data[].attributes.headline
Actual Work Compression What is it and Why It Is used?

# data[].attributes.subject
Actual Work Compression What is it and Why It Is used?

# data[].attributes.body
Really have no idea what compression is...

# data[].attributes.body
https://en.wikipedia.org/wiki/Lossless_compression

Lossless compression is a class of data compression algorithms that allows the original data to be perfectly reconstructed from the compressed data. By contrast, lossy compression permits reconstruction only of an approximation of the original data, though this usually improves compression rates (and therefore reduces file sizes).

Lossless data compression is used in many applications. For example, it is used in the ZIP file format and in the GNU tool gzip. It is also often used as a component within lossy data compression technologies (e.g. lossless mid/side joint stereo preprocessing by the LAME MP3 encoder and other lossy audio encoders).

Lossless compression is used in cases where it is important that the original and the decompressed data be identical, or where deviations from the original data would be unfavourable. Typical examples are executable programs, text documents, and source code. Some image file formats, like PNG or GIF, use only lossless compression, while others like TIFF and MNG may use either lossless or lossy methods. Lossless audio formats are most often used for archiving or production purposes, while smaller lossy audio files are typically used on portable players and in other cases where storage space is limited or exact replication of the audio is unnecessary.

# data[].attributes.body
In information technology, lossy compression or irreversible compression is the class of data encodingmethods that uses inexact approximations and partial data discarding to represent the content. These techniques are used to reduce data size for storage, handling, and transmitting content. Different versions of the photo of the cat above show how higher degrees of approximation create coarser images as more details are removed. This is opposed to lossless data compression (reversible data compression) which does not degrade the data. The amount of data reduction possible using lossy compression is much higher than through lossless techniques.

Well-designed lossy compression technology often reduces file sizes significantly before degradation is noticed by the end-user. Even when noticeable by the user, further data reduction may be desirable (e.g., for real-time communication, to reduce transmission times, or to reduce storage needs).

Lossy compression is most commonly used to compress multimedia data (audio, video, and images), especially in applications such as streaming media and internet telephony. By contrast, lossless compression is typically required for text and data files, such as bank records and text articles. In many cases it is advantageous to make a master lossless file which is to be used to produce new compressed files; for example, a multi-megabyte file can be used at full size to produce a full-page advertisement in a glossy magazine, and a 10 kilobyte lossy copy can be made for a small image on a web page.

Types[edit]

It is possible to compress many types of digital data in a way that reduces the size of a computer file needed to store it, or the bandwidth needed to transmit it, with no loss of the full information contained in the original file. A picture, for example, is converted to a digital file by considering it to be an array of dots and specifying the color and brightness of each dot. If the picture contains an area of the same color, it can be compressed without loss by saying "200 red dots" instead of "red dot, red dot, ...(197 more times)..., red dot."

The original data contains a certain amount of information, and there is a lower limit to the size of file that can carry all the information. Basic information theory says that there is an absolute limit in reducing the size of this data. When data is compressed, its entropy increases, and it cannot increase indefinitely. As an intuitive example, most people know that a compressed ZIP file is smaller than the original file, but repeatedly compressing the same file will not reduce the size to nothing. Most compression algorithms can recognize when further compression would be pointless and would in fact increase the size of the data.

In many cases, files or data streams contain more information than is needed for a particular purpose. For example, a picture may have more detail than the eye can distinguish when reproduced at the largest size intended; likewise, an audio file does not need a lot of fine detail during a very loud passage. Developing lossy compression techniques as closely matched to human perception as possible is a complex task. Sometimes the ideal is a file that provides exactly the same perception as the original, with as much digital information as possible removed; other times, perceptible loss of quality is considered a valid trade-off for the reduced data.

The terms 'irreversible' and 'reversible' are preferred over 'lossy' and 'lossless' respectively for some applications, such as medical image compression, to circumvent the negative implications of 'loss'. The type and amount of loss can affect the utility of the images. Artifacts or undesirable effects of compression may be clearly discernible yet the result still useful for the intended purpose. Or lossy compressed images may be 'visually lossless', or in the case of medical images, so-called Diagnostically Acceptable Irreversible Compression (DAIC)[1] may have been applied.

Transform coding[edit]

Main article: Transform coding

More generally, some forms of lossy compression can be thought of as an application of transform coding – in the case of multimedia data, perceptual coding: it transforms the raw data to a domain that more accurately reflects the information content. For example, rather than expressing a sound file as the amplitude levels over time, one may express it as the frequency spectrum over time, which corresponds more accurately to human audio perception. While data reduction (compression, be it lossy or lossless) is a main goal of transform coding, it also allows other goals: one may represent data more accurately for the original amount of space[2] – for example, in principle, if one starts with an analog or high-resolution digital master, an MP3 file of a given size should provide a better representation than a raw uncompressed audio in WAV or AIFF file of the same size. This is because uncompressed audio can only reduce file size by lowering bit rate or depth, whereas compressing audio can reduce size while maintaining bit rate and depth. This compression becomes a selective loss of the least significant data, rather than losing data across the board. Further, a transform coding may provide a better domain for manipulating or otherwise editing the data – for example, equalization of audio is most naturally expressed in the frequency domain (boost the bass, for instance) rather than in the raw time domain.

From this point of view, perceptual encoding is not essentially about discarding data, but rather about a better representation of data. Another use is for backward compatibility and graceful degradation: in color television, encoding color via a luminance-chrominance transform domain (such as YUV) means that black-and-white sets display the luminance, while ignoring the color information. Another example is chroma subsampling: the use of color spaces such as YIQ, used in NTSC, allow one to reduce the resolution on the components to accord with human perception – humans have highest resolution for black-and-white (luma), lower resolution for mid-spectrum colors like yellow and green, and lowest for red and blues – thus NTSC displays approximately 350 pixels of luma per scanline, 150 pixels of yellow vs. green, and 50 pixels of blue vs. red, which are proportional to human sensitivity to each component.

Information loss[edit]

Lossy compression formats suffer from generation loss: repeatedly compressing and decompressing the file will cause it to progressively lose quality. This is in contrast with lossless data compression, where data will not be lost via the use of such a procedure. Information-theoretical foundations for lossy data compression are provided by rate-distortion theory. Much like the use of probability in optimal coding theory, rate-distortion theory heavily draws on Bayesian estimation and decision theory in order to model perceptual distortion and even aesthetic judgment.

There are two basic lossy compression schemes:

In lossy transform codecs, samples of picture or sound are taken, chopped into small segments, transformed into a new basis space, and quantized. The resulting quantized values are then entropy coded.

In lossy predictive codecs, previous and/or subsequent decoded data is used to predict the current sound sample or image frame. The error between the predicted data and the real data, together with any extra information needed to reproduce the prediction, is then quantized and coded.

In some systems the two techniques are combined, with transform codecs being used to compress the error signals generated by the predictive stage.

Comparison[edit]

The advantage of lossy methods over lossless methods is that in some cases a lossy method can produce a much smaller compressed file than any lossless method, while still meeting the requirements of the application. Lossy methods are most often used for compressing sound, images or videos. This is because these types of data are intended for human interpretation where the mind can easily "fill in the blanks" or see past very minor errors or inconsistencies – ideally lossy compression is transparent (imperceptible), which can be verified via an ABX test. Data files using lossy compression are smaller in size and thus cost less to store and to transmit over the Internet, a crucial consideration for streaming video services such as Netflix and streaming audio services such as Spotify.

Transparency[edit]

Further information: Transparency (data compression)

When a user acquires a lossily compressed file, (for example, to reduce download time) the retrieved file can be quite different from the original at the bit level while being indistinguishable to the human ear or eye for most practical purposes. Many compression methods focus on the idiosyncrasies of human physiology, taking into account, for instance, that the human eye can see only certain wavelengths of light. The psychoacoustic model describes how sound can be highly compressed without degrading perceived quality. Flaws caused by lossy compression that are noticeable to the human eye or ear are known as compression artifacts.

Compression ratio[edit]

The compression ratio (that is, the size of the compressed file compared to that of the uncompressed file) of lossy video codecs is nearly always far superior to that of the audio and still-image equivalents.

Video can be compressed immensely (e.g. 100:1) with little visible quality loss

Audio can often be compressed at 10:1 with imperceptible loss of quality

Still images are often lossily compressed at 10:1, as with audio, but the quality loss is more noticeable, especially on closer inspection.

Transcoding and editing[edit]

For more details on this topic, see Transcoding.

An important caveat about lossy compression (formally transcoding), is that editing lossily compressed files causes digital generation loss from the re-encoding. This can be avoided by only producing lossy files from (lossless) originals and only editing (copies of) original files, such as images in raw image format instead of JPEG. If data which has been compressed lossily is decoded and compressed losslessly, the size of the result can be comparable with the size of the data before lossy compression, but the data already lost cannot be recovered. When deciding to use lossy conversion without keeping the original, one should remember that format conversion may be needed in the future to achieve compatibility with software or devices (format shifting), or to avoid paying patent royalties for decoding or distribution of compressed files.

Editing of lossy files[edit]

See also: commons:Commons:Software § JPEG, and commons:Commons:Software § Ogg Vorbis (audio)

By modifying the compressed data directly without decoding and re-encoding, some editing of lossily compressed files without degradation of quality is possible. Editing which reduces the file size as if it had been compressed to a greater degree, but without more loss than this, is sometimes also possible.

JPEG[edit]

The primary programs for lossless editing of JPEGs are jpegtran, and the derived exiftran (which also preserves Exif information), and Jpegcrop (which provides a Windows interface).

These allow the image to be

cropped

rotated, flipped, and flopped, or

converted to grayscale (by dropping the chrominance channel).

While unwanted information is destroyed, the quality of the remaining portion is unchanged.

Some other transforms are possible to some extent, such as joining images with the same encoding (composing side by side, as on a grid) or pasting images (such as logos) onto existing images (both via Jpegjoin), or scaling.[3]

Some changes can be made to the compression without re-encoding:

optimizing the compression (to reduce size without change to the decoded image)

converting between progressive and non-progressive encoding.

The freeware Windows-only IrfanView has some lossless JPEG operations in its JPG_TRANSFORM plugin.

Metadata[edit]

Metadata, such as ID3 tags, Vorbis comments, or Exif information, can usually be modified or removed without modifying the underlying data.

Downsampling/compressed representation scalability[edit]

One may wish to downsample or otherwise decrease the resolution of the represented source signal and the quantity of data used for its compressed representation without re-encoding, as in bitrate peeling, but this functionality is not supported in all designs, as not all codecs encode data in a form that allows less important detail to simply be dropped. Some well-known designs that have this capability include JPEG 2000 for still images and H.264/MPEG-4 AVC based Scalable Video Coding for video. Such schemes have also been standardized for older designs as well, such as JPEG images with progressive encoding, and MPEG-2 and MPEG-4 Part 2 video, although those prior schemes had limited success in terms of adoption into real-world common usage. Without this capacity, which is often the case in practice, to produce a representation with lower resolution or lower fidelity than a given one, one needs to start with the original source signal and encode, or start with a compressed representation and then decompress and re-encode it (transcoding), though the latter tends to cause digital generation loss.

Another approach is to encode the original signal at several different bitrates, and their either choose which to use (as when streaming over the internet – as in RealNetworks' "SureStream" – or offering varying downloads, as at Apple's iTunes Store), or broadcast several, where the best that is successfully received is used, as in various implementations of hierarchical modulation. Similar techniques are used in mipmaps, pyramid representations, and more sophisticated scale space methods. Some audio formats feature a combination of a lossy format and a lossless correction which when combined reproduce the original signal; the correction can be stripped, leaving a smaller, lossily compressed, file. Such formats include MPEG-4 SLS (Scalable to Lossless), WavPack, OptimFROG DualStream, and DTS-HD Master Audio in lossless (XLL) mode

Methods[edit]

Graphics[edit]

Image[edit]

Further information: Image compression

Better Portable Graphics, also known as BPG (lossless or lossy compression)

Cartesian Perceptual Compression, also known as CPC

DjVu

Fractal compression

ICER, used by the Mars Rovers, related to JPEG 2000 in its use of wavelets

JBIG2 (lossless or lossy compression)

JPEG

JPEG 2000, JPEG's successor format that uses wavelets (lossless or lossy compression)

JPEG XR, another successor of JPEG with support for high dynamic range, wide gamut pixel formats (lossless or lossy compression)

PGF, Progressive Graphics File (lossless or lossy compression)

S3TC texture compression for 3D computer graphics hardware

Wavelet compression

Video[edit]

Further information: Video compression

Motion JPEG

MPEG-1 Part 2

MPEG-2 Part 2

MPEG-4 Part 2

H.264/MPEG-4 AVC (may also be lossless, even in certain video sections)

Ogg Theora (noted for its lack of patent restrictions)

Dirac

Sorenson video codec

VC-1

H.265/HEVC

Audio[edit]

Further information: Audio data compression

Opus (mostly for real-time applications)

Music[edit]

AAC

ADPCM

ATRAC

Dolby Digital (AC-3)

MP2

MP3

Musepack (based on Musicam)

Ogg Vorbis (noted for its lack of patent restrictions)

WMA (Lossless codec available too)

Speech[edit]

Further information: Speech encoding

Adaptive Multi-Rate (Used in GSM and 3GPP)

Codec2 (noted for its lack of patent restrictions)

Speex (noted for its lack of patent restrictions)

Other data[edit]

Researchers have (semi-seriously) performed lossy compression on text by either using a thesaurus to substitute short words for long ones, or generative texttechniques,[4] although these sometimes fall into the related category of lossy data conversion.

Lowering resolution[edit]

A general kind of lossy compression is to lower the resolution of an image, as in image scaling, particularly decimation. One may also remove less "lower information" parts of an image, such as by seam carving. Many media transforms, such as Gaussian blur, are, like lossy compression, irreversible: the original signal cannot be reconstructed from the transformed signal. However, in general these will have the same size as the original, and are not a form of compression. Lowering resolution has practical uses, as the NASA New Horizons craft will transmit thumbnails of its encounter with Pluto-Charon before it sends the higher resolution images. Another solution for slow connections is the usage of Image interlacing which progressively defines the image. Thus a partial transmission is enough to preview the final image, in a lower resolution version, without creating a scaled and a full version too.

See also[edit]

Data compression

Lossless compression

Compression artifact

Rate–distortion theory

List of codecs

Lenna

Image scaling

Seam carving

Transcoding

Notes[edit]

1. Jump up^ European Society of Radiology (2011). "Usability of irreversible image compression in radiological imaging. A position paper by the European Society of Radiology (ESR)". Insights Imaging. 2: 103–115. PMC 3259360 . PMID 22347940. doi:10.1007/s13244-011-0071-x.

2. Jump up^ “Although one main goal of digital audio perceptual coders is data reduction, this is not a necessary characteristic. As we shall see, perceptual coding can be used to improve the representation of digital audio through advanced bit allocation.” Masking and Perceptual Coding, Victor Lombardi

3. Jump up^ New jpegtran features

4. Jump up^ I. H. WITTEN; et al. "Semantic and Generative Models for Lossy Text Compression" (PDF). The Computer Journal. Retrieved 2007-10-13.

External links[edit]

Lossy audio formats, comparing the speed and compression strength of five lossy audio formats.

Data compression basics, including chapters on lossy compression of images, audio and video.

Lossy PNG image compression (research)

Using lossy GIF/PNG compression for the web (article)

JPG for Archiving, comparing the suitability of JPG and lossless compression for image archives

[hide]

v

t

e

Data compression methods

Lossless | Entropy type | Unary Arithmetic Asymmetric numeral systems Golomb Huffman Adaptive Canonical Modified Range Shannon Shannon–Fano Shannon–Fano–Elias Tunstall Universal Exp-Golomb Fibonacci Gamma Levenshtein
Dictionary type | Byte pair encoding DEFLATE Snappy Lempel–Ziv LZ77 / LZ78 (LZ1 / LZ2)LZFSE LZJB LZMA LZO LZRW LZS LZSS LZW LZWL LZX LZ4 Brotli Zstandard
Other types | BWT CTW Delta DMC MTF PAQ PPM RLE
Audio | Concepts | Bit rate average (ABR)constant (CBR)variable (VBR) Companding Convolution Dynamic range Latency Nyquist–Shannon theorem Sampling Sound quality Speech coding Sub-band coding
Codec parts | A-law μ-law ACELP ADPCM CELP DPCM Fourier transform LPC LAR LSP MDCT Psychoacoustic model WLPC
Image | Concepts | Chroma subsampling Coding tree unit Color space Compression artifact Image resolution Macroblock Pixel PSNR Quantization Standard test image
Methods | Chain code DCT EZW Fractal KLT LP RLE SPIHT Wavelet
Video | Concepts | Bit rate average (ABR)constant (CBR)variable (VBR) Display resolution Frame Frame rate Frame types Interlace Video characteristics Video quality
Codec parts | Lapped transform DCT Deblocking filter Motion compensation
Theory | Entropy Kolmogorov complexity Lossy Quantization Rate–distortion Redundancy Timeline of information theory Compression formats Compression software (codecs)

Categories:

Data compression

Lossy compression algorithms

Navigation menu

# data[].attributes.body
Data compression is a reduction in the number of bits needed to represent data. Compressing data can save storage capacity, speed up file transfer, and decrease costs for storage hardware and network bandwidth.

# data[].attributes.body
https://en.wikipedia.org/wiki/Lossy_compression

# data[].attributes.body
http://searchstorage.techtarget.com/definition/compression

How compression works

Compression is performed by a program that uses a formula or algorithm to determine how to shrink the size of the data. For instance, an algorithm may represent a string of bits -- or 0s and 1s -- with a smaller string of 0s and 1s by using a dictionary for the conversion between them, or the formula may insert a reference or pointer to a string of 0s and 1s that the program has already seen.

Text compression can be as simple as removing all unneeded characters, inserting a single repeat character to indicate a string of repeated characters and substituting a smaller bit string for a frequently occurring bit string. Data compression can reduce a text file to 50% or a significantly higher percentage of its original size.

For data transmission, compression can be performed on the data content or on the entire transmission unit, including header data. When information is sent or received via the internet, larger files, either singly or with others as part of an archive file, may be transmitted in a ZIP, GZIP or other compressed format.

Why is data compression important?

Data compression can dramatically decrease the amount of storage a file takes up. For example, in a 2:1 compression ratio, a 20 megabyte (MB) file takes up 10 MB of space. As a result of compression, administrators spend less money and less time on storage.

PRO+

Content

Find more PRO+ content and other member only offers, here.

E-Zine

2017 IT spending trends for data storage

E-Handbook

Take a deep dive into software-defined storage products

E-Zine

The state of flash storage performance

Compression optimizes backup storage performance and has recently shown up in primary storage data reduction. Compression will be an important method of data reduction as data continues to grow exponentially.

Virtually any type of file can be compressed, but it's important to follow best practices when choosing which ones to compress. For example, some files may already come compressed, so compressing those files would not have a significant impact.

Data compression methods: lossless and lossy compression

Compressing data can be a lossless or lossy process. Lossless compression enables the restoration of a file to its original state, without the loss of a single bit of data, when the file is uncompressed. Lossless compression is the typical approach with executables, as well as text and spreadsheet files, where the loss of words or numbers would change the information.

Lossy compression permanently eliminates bits of data that are redundant, unimportant or imperceptible. Lossy compression is useful with graphics, audio, video and images, where the removal of some data bits has little or no discernible effect on the representation of the content.

Professor David Brailsford, with the School of
Computer Science at the University of Nottingham,
discusses compression of text and pictures.

Graphics image compression can be lossy or lossless. Graphic image file formats are typically designed to compress information since the files tend to be large. JPEG is an image file format that supports lossy image compression. Formats such as GIF and PNG use lossless compression.