Sorting grouped multi-dimensional array by value

$data = array(
    'apple' => array(
        0 => array('sort'=>4, 'name'=>'apple_4'),
        1 => array('sort'=>10, 'name'=>'apple_10'),
        2 => array('sort'=>5, 'name'=>'apple_5'),
        3 => array('sort'=>1, 'name'=>'apple_1')
        ),

    'orange' => array(
        0 => array('sort'=>4, 'name'=>'orange_4'),
        1 => array('sort'=>10, 'name'=>'orange_10')
        )
    );

Need assistance sorting multi-dimensional array. For the array above, I would like to sort the contents of each group in descending order by the 'sort' value. The group's keys should remain in tact (apple, orange) but content's keys are not important.

Data should be ordered:

• apple
  • apple_10
  • apple_5
  • apple_4
  • apple_1
• orange
  • orange_10
  • orange_4