• DRC's avatar
    Fix block smoothing w/vert.-subsampled prog. JPEGs · 9b704f96
    DRC authored
    The 5x5 interblock smoothing implementation, introduced in libjpeg-turbo
    2.1, improperly extended the logic from the traditional 3x3 smoothing
    implementation.  Both implementations point prev_block_row and
    next_block_row to the current block row when processing, respectively,
    the first and the last block row in the image:
    
      if (block_row > 0 || cinfo->output_iMCU_row > 0)
        prev_block_row =
          buffer[block_row - 1] + cinfo->master->first_MCU_col[ci];
      else
        prev_block_row = buffer_ptr;
    
      if (block_row < block_rows - 1 ||
          cinfo->output_iMCU_row < last_iMCU_row)
        next_block_row =
          buffer[block_row + 1] + cinfo->master->first_MCU_col[ci];
      else
        next_block_row = buffer_ptr;
    
    6d91e950 naively extended that logic to
    accommodate a 5x5 smoothing window:
    
      if (block_row > 1 || cinfo->output_iMCU_row > 1)
        prev_prev_block_row =
          buffer[block_row - 2] + cinfo->master->first_MCU_col[ci];
      else
        prev_prev_block_row = prev_block_row;
    
      if (block_row < block_rows - 2 ||
          cinfo->output_iMCU_row + 1 < last_iMCU_row)
        next_next_block_row =
          buffer[block_row + 2] + cinfo->master->first_MCU_col[ci];
      else
        next_next_block_row = next_block_row;
    
    However, this new logic was only correct if block_rows == 1, so the
    values of prev_prev_block_row and next_next_block_row were incorrect
    when processing, respectively, the second and second to last iMCU rows
    in a vertically-subsampled progressive JPEG image.
    
    The intent was to:
    - point prev_block_row to the current block row when processing the
      first block row in the image,
    - point prev_prev_block_row to prev_block_row when processing the first
      two block rows in the image,
    - point next_block_row to the current block row when processing the
      last block row in the image, and
    - point next_next_block_row to next_block_row when processing the last
      two block rows in the image.
    
    This commit modifies decompress_smooth_data() so that it computes the
    current block row's position relative to the whole image and sets
    the block row pointers based on that value.
    
    This commit also restores a line of code that was accidentally deleted
    by 6d91e950:
    
      access_rows += compptr->v_samp_factor; /* prior iMCU row too */
    
    access_rows is merely a sanity check that tells the access_virt_barray()
    method to generate an error if accessing the specified number of rows
    would cause a buffer overrun.  Essentially, it is a belt-and-suspenders
    measure to ensure that j*init_d_coef_controller() allocated enough rows
    for the full-image virtual array.  Thus, excluding that line of code did
    not cause an observable issue.
    
    This commit also documents dbae5928 in
    the change log.
    
    Fixes #721
    9b704f96
To find the state of this project's repository at the time of any of these versions, check out the tags.