Non-stationarity, non-Gaussianity and temporal dependence are commonly encountered in large-scale structured data, emerging from scientific studies in neuroscience and meteorology among others. These challenging features may not fit into existing theoretical framework or data analysis tools. Motivated from the multi-scan multi-subject fMRI data analysis, this paper proposes a new semi-parametric inference procedure applicable to a broad class of “non-stationary non-Gaussian temporally dependent” noise processes for time-course data collected at spatial points. A new test statistic is developed based on a tapering-type estimator of the large-dimensional noise auto-covariance matrix, and its asymptotic chi-squared distribution is established. Our method benefits from avoiding directly inverting the noise covariance matrix without reducing efficiency, adaptive to either stationary or a wide class of non-stationary noise processes, thus is particularly effective in dealing with practically challenging cases arising from very large-scales of data and large-dimensions of covariance matrices. The efficacy of the proposed procedure over existing methods is demonstrated through simulation evaluations and real fMRI data analysis.