RSS
 

Posts Tagged ‘compression’

NVIDIA Research develops a neural network to replace traditional video compression

06 Oct

NVIDIA researchers have demonstrated a new type of video compression technology that replaces the traditional video codec with a neural network to drastically reduce video bandwidth. The technology is presented as a potential solution for streaming video in situations where Internet availability is limited, such as using a webcam to chat with clients while on a slow Internet connection.

The new technology is made possible using NVIDIA Maxine, a cloud-AI video streaming platform for developers. According to the researchers, using AI-based video compression can strip video bandwidth usage down to 1/10th of the bandwidth that would otherwise be used by the common H.264 video codec. For users, this could result in what NVIDIA calls a ‘smoother’ experience that uses up less mobile data.

In a video explaining the technology, researchers demonstrate their AI-based video compression alongside H.264 compression with both videos limited to the same low bandwidth. With the traditional video compression, the resulting low-bandwidth video is very pixelated and blocky, but the AI-compressed video is smooth and relatively clear.

This is made possible by extracting the key facial points on the subject’s face, such as the position of the eyes and mouth, then sending that data to the recipient. The AI technology then reconstructs the subject’s face and animates it in real time using the keypoint data, the end result being very low bandwidth usage compared to the image quality on the receiver’s end.

There are some other advantages to using AI-based compression that exceed the capabilities of traditional video technologies, as well. One example is Free View, a feature in which the AI platform can rotate the subject so that they appear to be facing the recipient even when, in reality, their camera is positioned off to the side and they appear to be staring into the distance.

Likewise, the keypoints extracted from the subject’s face could also be used to apply their movements to other characters, including fully animated characters, expanding beyond the AI-powered filters that have become popular some video apps like Snapchat. Similar technology is already on the market in the form of Apple’s AI-based Animoji.

The use of artificial intelligence to modify videos isn’t new; most major video conferencing apps now include the option of replacing one’s real-life background with a different one, including intelligent AI-based background blurring. However, NVIDIA’s real-time AI-based video compression takes things to a new level by using AI to not only generate the subject in real time, but also modify them in convenient ways, such as aligning their face with a virtual front-facing camera.

The technology could usher in an era of clearer, more consistent video conferencing experiences, particularly for those on slow Internet connections, while using less data than current options. However, the demonstration has also raised concerns that largely mirror ones related to deepfake technologies — namely, the potential for exploiting such technologies to produce inauthentic content.

Artificial intelligence technology is advancing at a clipped rate and, in many cases, can be used to imperceptibly alter videos and images. Work is already underway to exceed those capabilities, however, by fully generating photo-realistic content using AI rather than modifying existing real-world content.

The Allen Institute for AI recently demonstrated the latest evolution in this effort by using both images and text to create a machine learning algorithm that possesses a very basic sense of abstract reasoning, for example. NVIDIA Research has also contributed extensively to this rapidly evolving technology, with past demonstrations including generating landscapes from sketches, generating photo-realistic portraits and even swapping facial expressions between animals.

A number of companies are working to develop counter technologies capable of detecting manipulated content by looking for markers otherwise invisible to the human eye. In 2019, Adobe Research teamed up with UC Berkeley to develop and demonstrate an AI capable of not only identifying portrait manipulations, but also automatically reversing the changes to display the original, unmodified content.

The general public doesn’t yet have access to these types of technologies, however, generally leaving them vulnerable to the manipulated media that permeates social media.

Via: NVIDIA

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on NVIDIA Research develops a neural network to replace traditional video compression

Posted in Uncategorized

 

New H.266/VVC video compression standard will reduce video sizes by up to 50%

09 Jul

A new video encoding standard that reduces video file sizes by 50% is set to become usable by the end of the year, allowing high-resolution footage to be saved with smaller file sizes and to be transmitted in less time. H.266/VVC (Versatile Video Coding) follows on from the current H.265 compression standard and was developed by Fraunhofer HHI alongside companies such as Sony, Apple, Intel, Huawei, Microsoft, Qualcomm and Ericsson.

The idea of the new standard is to compress files so that 4K and 8K footage become easier and quicker to move, particularly over slower network connections. The effect should be that all video footage takes up less space on a memory card and can be copied over to a hard disk in less time, all while using less computer power. That footage can also be posted online, to external storage, sent to a third party or streamed more quickly/easily due to the reduced file size.

The new H.266 standard will also allow systems or locations with poor data rates to receive larger files more quickly, so movies, for example, will buffer less and play more smoothly. Mobile devices will also be able to send higher resolution files, or longer clips, without using so much data.

Fraunhofer says that ‘H.266/VVC offers faster video transmission for equal perceptual quality,’ so we shouldn’t see the difference between files compressed by H.266 and those compressed using H.265. H.265 also halved file sizes when it was introduced, as did the H.264 standard that came before that — and which is still in use today.

The new standard requires new chips to make the most of it, and the press release states that they are already in production and that Fraunhofer will release the software to allow the standard to be used in the autumn of this year. For more information see the Fraunhofer website.

Press release:

Fraunhofer HHI is proud to present the new state-of-the-art in global video coding: H.266/VVC brings video transmission to new speed

After devoting several years to its research and standardization, Fraunhofer HHI (together with partners from industry including Apple, Ericsson, Intel, Huawei, Microsoft, Qualcomm, and Sony) is celebrating the release and official adoption of the new global video coding standard H.266/Versatile Video Coding (VVC). This new standard offers improved compression, which reduces data requirements by around 50% of the bit rate relative to the previous standard H.265/High Efficiency Video Coding (HEVC) without compromising visual quality. In other words, H.266/VVC offers faster video transmission for equal perceptual quality. Overall, H.266/VVC provides efficient transmission and storage of all video resolutions from SD to HD up to 4K and 8K, while supporting high dynamic range video and omnidirectional 360° video.

Today, compressed video data make up 80% of global Internet traffic. H.266/VVC represents the pinnacle of (at least) four generations of international standards for video coding. The previous standards H.264/Advanced Video Coding (AVC) and H.265/HEVC, which were produced with substantial contributions from Fraunhofer HHI, remain active in more than 10 billion end devices, processing over 90% of the total global volume of video bits. Both previous standards were also recognized by collectively three Emmy Engineering Awards for contributing substantially to the progress of television technology.

Through a reduction of data requirements, H.266/VVC makes video transmission in mobile networks (where data capacity is limited) more efficient. For instance, the previous standard H.265/HEVC requires ca. 10 gigabytes of data to transmit a 90-min UHD video. With this new technology, only 5 gigabytes of data are required to achieve the same quality. Because H.266/VVC was developed with ultra-high-resolution video content in mind, the new standard is particularly beneficial when streaming 4K or 8K videos on a flat screen TV. Furthermore, H.266/VVC is ideal for all types of moving images: from high-resolution 360° video panoramas to screen sharing contents.

“After dedicating almost three years toward this standard, we are proud to have been instrumental in developing H.266/VVC,” says Benjamin Bross, head of the Video Coding Systems group at Fraunhofer HHI and editor of the +500-page standard specification of H.266/VVC. “Because of the quantum leap in coding efficiency offered by H.266/VVC, the use of video will increase further worldwide. Moreover, the increased versatility of H.266/VVC makes its use more attractive for a broader range of applications related to the transmission and storage of video.”

“If you consider that Fraunhofer HHI already played a key role in the development of the previous video coding standards H.264/AVC and H.265/HEVC, then we are happy with the fact that more than 50% of the bits on the Internet are generated by a Fraunhofer HHI technology,” adds Dr. Detlev Marpe, head of the Video Coding and Analytics department at Fraunhofer HHI.

A uniform and transparent licensing model based on the FRAND principle (i.e., fair, reasonable, and non-discriminatory) is planned to be established for the use of standard essential patents related to H.266/VVC. For this purpose, the Media Coding Industry Forum (MC-IF) was founded. In addition to Fraunhofer Society, the MC-IF now includes +30 companies and organizations. The new chips required for the use of H.266/VVC, such as those in mobile devices, are currently being designed. Dr. Thomas Schierl, head of the Video Coding and Analytics department at Fraunhofer HHI, announced “this autumn Fraunhofer HHI will publish the first software (for both encoder and decoder) to support H.266/VVC.”

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on New H.266/VVC video compression standard will reduce video sizes by up to 50%

Posted in Uncategorized

 

JPEG Committee explores using AI and blockchain tech for image compression

25 Feb

On February 17, the JPEG Committee published the results from its 86th Meeting, detailing some of the topics of discussions and potential future plans. Among other things, the committee issued a Call for Evidence on what it refers to as learning-based image coding solutions, something following the JPEG AI activity the committee launched a year ago. As well, the committee has expanded discussions on the use of blockchain technology and distributed ledger technologies (DLT) for JPEG.

During the 85th JPEG meeting last year, an effort dubbed JPEG AI was initiated in order to explore the use of image coding technologies to increase compression efficiency. During the new 86th JPEG meeting, this effort was expanded to a formal Call for Evidence, which is described as the first step in considering the ‘standardization of such approaches in image compression.’

In the JPEG AI Call for Evidence, JPEG Committee states:

‘This activity aims to find evidence for image coding technologies that offer substantially better compression efficiency than available image codecs with models obtained from a large amount of visual data and that can efficiently represent the wide variety of visual content that is available nowadays.’

In addition, the most recent meeting featured an Open Discussion Session on Media Blockchain that involved ‘interactive discussions’ on media blockchain and its various uses, namely its suitability for addressing ‘challenges in transparent and trustable media transactions.’ The committee has shared the presentations and pitch slides from these discussions on its website.

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on JPEG Committee explores using AI and blockchain tech for image compression

Posted in Uncategorized

 

Twitter rolls out Live Photo support on iOS, improved JPEG compression

13 Dec

Twitter has updated its platform with two new useful features for photographers: iOS Live Photos support and better JPEG quality. Both changes are live now.

Live Photos are a type of image that can be captured using an iPhone or iPad; in addition to the image, Live Photos include the 1.5 seconds of action that happened before and after the photo was snapped. In order to make it possible to share these images, Twitter is first converting them into GIFs.

To share a Live Photo, iPhone users must launch the Twitter mobile app and select the image from their Camera Roll. Once the Live Photo is selected, the user can tap the new ‘GIF’ option located in the bottom left corner of the image. This will result in Twitter converting and sharing the Live Photo as a GIF.

Converting Live Photos into GIFs has been the primary method used to share the video versions of these images. Lack of direct support on many platforms has forced many iPhone users to turn to apps like Lively. Twitter’s new support merely removes this time-consuming manual conversion process, enabling iPhone users to rapidly share their Live Photos with followers.

In addition to the new direct Live Photos to GIF conversion feature, Twitter is also now publishing JPEGs with their original encoding, according to company engineer Nolan O’Brien.This eliminates the transcoding and compression that obliterates image quality when viewed in full size. O’Brien notes that the thumbnail version of JPEGs will still be transcoded to cut down on file size and that only the bitmap encoding is preserved, not the metadata. As well, the new encoding preservation is only live for images uploaded using Twitter for Web.

Twitter for Web has supported 4096 x 4096 image uploads since last year, according to O’Brien, who details some upload scenarios in which the platform will still encode images:

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on Twitter rolls out Live Photo support on iOS, improved JPEG compression

Posted in Uncategorized

 

Video: There’s no such thing as lens compression, it’s just perspective distortion

24 May

We’ve been saying for years that the term “lens compression” is misleading, but Lee Morris over at Fstoppers has put together a useful video that explains exactly why this is the case, and demonstrates it with two easy-to-understand examples.

The main issue with the term “lens compression” is that the distortion the term refers to has nothing to do with the lens itself. The issue is simply perspective distortion, caused by the distance between your camera and your subject, as well as the distance between your camera and the background.

Put another way: if your subject is 1 meter away (or feet: it doesn’t really matter), and your background is 50 meters away, moving back 1 meter will double the distance between you and your subject, while barely changing the distance between you and the background—the perspective on your subject changes drastically, while the perspective on your background barely shifts at all.

This diagram, from the FStoppers video, shows why changing your perspective appears to compress the background… When you double the distance to your subject you halve its size, but you’ve barely moved in relation to the background, so it remains roughly the same size in your image.

To show this concept in action, Morris uses two examples. First, he shows you how you can get the exact same perspective using a 24mm lens that you can with a 400mm lens by simply cropping the wide-angle shot. Then, he does the opposite, creating the same perspective as a 15mm shot by stitching multiple shots taken at 70mm.

Of course, that doesn’t mean you should go throw out all of your lenses and just pick one focal length to either crop or stitch with. Physical limitations apply: like how much room you have to back up, how much resolution you’re willing to sacrifice by cropping, and how much sanity you have to spare if you’re trying to create a 15mm shot by taking a thousand shots with an 800mm lens.

The demonstration is just that: a demonstration of a concept that is often misunderstood because of the language we use to describe it. The compression you get using a long lens isn’t a result of the lens, so much as the distance between your subject, your background, and the camera.

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on Video: There’s no such thing as lens compression, it’s just perspective distortion

Posted in Uncategorized

 

Google uses neural networks to improve image compression

27 Aug

A research team at Google has developed a way to use neural networks to compress image files in a more efficient way than current methods, such as the JPEG standard. The team built an artificial intelligence system using Google’s open source TensorFlow machine learning system, and then used 6 million random reference photos from the internet that had been compressed using conventional methods to train it.

The images were split into small pieces measuring 32 x 32 pixels each. The system then analyzed the 100 pieces with the least efficient compression; the idea being that it could learn from looking at the most complex areas of an image, making compression of less complex sections much easier.

After the initial training process the AI system is then able to predict how the image would look like after compression and then generates that image. What makes this method really stand out from others is that the network can intelligently decide which is the best way to compress individual areas of a given photo for the best overall result. The method still needs some work, as final results can sometimes look unpleasant to the human eye and the system are not yet capable of testing for this. Nevertheless, the project looks like an important step into the right direction and if the algorithms can be further refined you might soon be able to save even more images on your memory card or built-in device storage.

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on Google uses neural networks to improve image compression

Posted in Uncategorized

 

Armadillo Vault: Delicate Stone Canopy Held Together by Compression

02 Jun

[ By SA Rogers in Architecture & Public & Institutional. ]

stone canopy 1

Not a drop of glue or any other adhesive holds together the delicate, 2-inch-thick limestone tiles that make up this airy canopy, which billows into the vaulted ceilings of the Arsenale di Venezia at the 15th annual Venice Architecture Biennale. Conceived by Block Research Group and presented by ETH Zurich, the ‘Armadillo Vault’ is a temporary custom-built installation showing off the surprising versatility of an unyielding material that’s been an architectural mainstay for millennia. Compression keeps all 399 individually-cut, unreinforced stones in place as they stretch across the cavernous space.

stone canopy 2

stone canopy 3

The centerpiece of an installation entitled ‘Beyond Bending – Learning from the past to design a better future,’ the Armadillo Vault aims to show the world that digital design and fabrication methods can go hand in hand with humble, ancient building materials like earth and stone. Other components on display include four innovative vaulted floor systems and a series of graphical force diagrams showing how the stones fit together.

stone canopy 4

stone canopy 5

Spanning over 52 feet through the Arsenale, the canopy was initially manufactured and assembled in Texas before being disassembled and shipped to Venice, with master stonemasons tasked with setting it into place on-site over a period of two weeks. The precision of the puzzle-like assembly to hold up all that weight with very few supports is a result of mathematically analyzing the structure to control compressive forces.

stone canopy 6

 

Check out how the structure was designed and installed in this video from Block Research Group, including computer models that show off the tessellation and Voussoir Geometry used to design the tile assembly.

Share on Facebook





[ By SA Rogers in Architecture & Public & Institutional. ]

[ WebUrbanist | Archives | Galleries | Privacy | TOS ]


WebUrbanist

 
Comments Off on Armadillo Vault: Delicate Stone Canopy Held Together by Compression

Posted in Creativity

 

How to use Focal Length and Background Compression to Enhance Your Photos

17 Nov

One of the most common uses for zoom lenses is, as their name suggests, to zoom in on objects that are far away. These lenses are fantastic for getting close-up views of nature, architecture, wildlife, or anything else that might be little more than a speck to the naked human eye.

Some cameras like the Nikon Coolpix P900 let you get a close-up view of objects a few miles away. While this flexibility might seem like a rather tempting proposition for getting close to objects without physically moving yourself, there is another often-overlooked benefit of zoom lenses when taking portraits or other types of pictures with one clear subject in front of a vast spread of scenery – background compression. Understanding how this works, and how you can manipulate it, can transform your approach to portrait photography and give your pictures the type of visual boost that you might have always wanted, but never knew how to achieve.

background-compression-senior-photo

The basic idea with background compression is that you can take photos of something relatively close to you, such as a high school senior as shown in the image above, and bring elements of the background closer as well. This gives a more constrained feeling to the overall composition, and helps focus the viewer on your subject while not only bringing the background in, but often blurring it at the same time.

As an illustration of how this works, here are several photos of my dad taken at different focal lengths. Notice how he is framed similarly in each shot, but the background changes dramatically as I adjust the zoom on my camera lens.

18mm focal length, f/7.1, 1/80th of a second, ISO 100

I used an 18-270mm zoom lens to take these shots, and this first one (above) at 18mm shows my dad along with a massive background: utility poles, houses, trees, mailboxes, and all sorts of other elements make up the picture in addition to the subject. Take note of the car several hundred yards behind him, as indicated by the red arrow, and notice what happens as I change focal lengths, but keep my dad similarly framed.

18mm, f/7.1, 1/125 second, ISO 320

70mm, f/7.1, 1/125 second, ISO 320

Here you can already see several differences from the original image. The scene is now slightly claustrophobic with many of the elements along the perimeter of the original photo disappearing altogether. Mailboxes and utility poles have been brought closer, and notice how the same stationary automobile far in the distance has appeared to creep forward, and is now much larger. The background, in essence, is getting squeezed together or compressed.

154mm, f/7.1, 1/250 second, ISO 800

154mm, f/7.1, 1/250 second, ISO 800

At 154mm the vehicle in the background seems significantly closer, and various other elements such as trees and utility poles are now filling almost the entire frame. As I zoom in, while keeping my dad consistently framed in the shot, even the distance between the individual utility poles seems to be shortened, which further enhances the overall feeling of compression. It’s not just that things appear closer, but that the distances between all the elements of the frame look much smaller as well. This can be a powerful, and extraordinarily useful way to compose a picture, and you don’t even need a fancy camera or lens to do it. Most pocket cameras have optical zooms that can be used to accomplish the same effect.

270mm, f/7.1, 1/400 second, ISO 1600

270mm, f/7.1, 1/400 second, ISO 1600

In this final shot, the background elements virtually dominate the frame and almost overpower my subject. The vehicle just over his shoulder is about a quarter of a mile (0.4 km) away, though it appears as if it’s a mere stone’s throw behind him.

Background compression can be a good or bad thing, depending on the type of picture you are taking. The key takeaway here is to know what it is, and how to utilize it to get the type of composition you are going for. The longer your focal length, the more you will be able to add this sense of compression to your background. But, it also helps if you have a great deal of distance between your subject and the elements behind it. If my dad were standing a few feet away from something, like a tree or a brick wall, there would be virtually no compression at all, even with a very long focal length.

18mm, f/7.1, 1/80 second, ISO 100

18mm lens

18mm, f/7.1, 1/125 second, ISO 320

70mm lens

background-compression-dad-154mm-f71

154mm lens

background-compression-dad-270mm-f71

270mm lens

If the final example in the above series seems a bit extreme, here’s another set of images that show how background compression can be used effectively to enhance the overall composition, rather than overpower your subject.

35mm, f/2.8, 1/750 second, ISO 200

35mm, f/2.8, 1/750 second, ISO 200

This is a perfectly serviceable portrait, although I purposely left a bit too much space on the right-hand side in order to illustrate the compression concept. Shot at 35mm, with a wide aperture of f/2.8, the background is nice and blurry, the focus is clearly on the smiling woman, and the background is not too distracting or bothersome. However, re-taking the same picture with a longer focal length, leads to a much more pleasing picture all around.

85mm, f/2.8, 1/750 second, ISO 200

85mm, f/2.8, 1/750 second, ISO 200

The same compression shown in the first series of pictures is clearly evident here, though it is used to a much better overall effect. Even though the tree and cars create a background that is somewhat busy, shooting with a wide aperture blurred things out enough, that the focus is still clearly on the woman, while the background serves to add a bit of context to help put the overall composition in perspective.

One final note about compression: it doesn’t just work for foreground elements too as you can see in the following pictures.

35mm, f/11, 1/125 second, ISO 400

35mm, f/11, 1/125 second, ISO 400

I purposely shot this with a much smaller aperture in order to minimize the degree of background bokeh, lest compression be confused with blur. Notice how this woman is sitting squarely in the middle of the bench with plenty of room to her right, and about 50 yards (46 meters) between her, and the trees and cars in the background. Most of the picture is in focu,s which is a direct result of the small aperture.

85mm, f/11, 1/90 second, ISO 400

85mm, f/11, 1/90 second, ISO 400

Here the foreground and background have both been brought nearer to the subject. The trees and cars behind her are much closer, while the bench appears to take up almost no room on the woman’s right side. It might as well be little more than a chair in this picture, and yet, this is merely an illusion created by using a longer focal length while keeping my subject framed appropriately. Most of the picture remains in focus due to the small aperture, and you can clearly see that background compression is not always synonymous with background blur.

As one final example, here is the same woman, on the same bench, shot wide open at f/1.8 with my 85mm lens.

85mm, f/1.8, 1/1000 second, ISO 400

85mm, f/1.8, 1/1000 second, ISO 400

The overall compositional elements remain the same in this final image as the two above, except that I moved myself physically closer to the subject, while shooting at a very wide aperture of f/1.8. The background is severely compressed, and quite blurry, which leads to a rather pleasing portrait.

Background compression can be a bit tricky to understand, but if you play around with different focal lengths you should get the hang of it quickly. Then it’s just a matter of figuring out how to use it to your advantage to create the type of shot you want–especially when doing portraits.

Have you tried using this technique in your own photography? What other tips do you have to share about creative uses for background and foreground compression? Leave your thoughts in the comments below, and feel free to share any example images you have as well.

This week on dPS we’re featuring a series of articles about composition. Many different elements and ways to compose images for more impact. Check out the ones we’ve done so far:

  • Using Framing for More Effective Compositions
  • 7 Tips to Improve Your Skyline Photos
  • 33 Images that Exemplify Compositional Elements
  • Weekly Photography Challenge – Composition Craziness
  • How to Take Control of Aperture and Create Stronger Photos
  • How Cropping in Post-Production Can Improve Composition
  • Good Crop Bad Crop – How to Crop Portraits

googletag.cmd.push(function() {
tablet_slots.push( googletag.defineSlot( “/1005424/_dPSv4_tab-all-article-bottom_(300×250)”, [300, 250], “pb-ad-78623” ).addService( googletag.pubads() ) ); } );

googletag.cmd.push(function() {
mobile_slots.push( googletag.defineSlot( “/1005424/_dPSv4_mob-all-article-bottom_(300×250)”, [300, 250], “pb-ad-78158” ).addService( googletag.pubads() ) ); } );

The post How to use Focal Length and Background Compression to Enhance Your Photos by Simon Ringsmuth appeared first on Digital Photography School.


Digital Photography School

 
Comments Off on How to use Focal Length and Background Compression to Enhance Your Photos

Posted in Photography

 

The Raw and the cooked: pulling apart Sony’s Raw compression

02 Sep

A Raw file is a Raw file, right? Well, not exactly. Lately, there’s been a lot of talk and a lot of anger about the compression Sony uses in its Raw files. Compressed Raw files aren’t uncommon, but they’re usually compressed in a way that retains all the original ‘raw’ data from the sensor. That doesn’t seem to be the case with Sony’s latest cameras. Read more

Articles: Digital Photography Review (dpreview.com)

 
Comments Off on The Raw and the cooked: pulling apart Sony’s Raw compression

Posted in Uncategorized