Place the milk in a heavy-based saucepan and bring up to boiling point. Remove from the heat and set aside.
Place the egg yolks, sugar and cornflour in a bowl and mix until creamy. Pour on the milk and mix until smooth.
Return to the rinsed-out saucepan and cook over medium heat, stirring continuously, until the custard is thick. Remove from the heat and beat well to remove any lumps that might have formed.
Place in a bowl and cover custard with baking paper to prevent a skin from forming. Leave to cool.
Once cool, fold in the mascarpone and then the whipped cream to lighten the custard. Whisk egg whites to soft peaks and fold into the custard mixture.
Pour the coffee and brandy into a shallow bowl. Dip about half the sponge fingers into this mixture, ensuring they do not become totally saturated, and arrange in the bottom of a large serving bowl.
Cover with half the custard mixture and sprinkle with half the chocolate. Repeat with the remaining sponge fingers, custard mixture and chocolate.
Cover with plastic wrap and refrigerate to firm up for at least 4 hours.
Before serving, top with the whipped cream and dust with the cocoa powder.