Unfortunately, anything placed in your detail band will be repeated with every detail band output row, unless you use a "Print When" expression to limit the output to a certain row number.
What if the left-side portion was actually a subreport, and you dropped that into an outer wrapper report. In the outer report, you could split the detail band into 2 columns and display the subreport with the data on the left. In that subreport, calculate your averages in the report footer but don't display them. Pass them back to the outer report as return values. You would be doing all of this with the goal of having only one row in the outer wrapper report's detail band.
Maybe someone else will have a better solution. I have had to get creative sometimes to achieve the layouts I'm looking for. For example, if you know you're always going to have X number of rows in your data table (I do a lot of "this month, last month, and the month before that" in my reports), you could use something like Oracle's decode function (not sure what kind of database you're using) to get all of the data in one row, and then you can arrange your text fields as you please within the single output row.