Global S&T Development Trend Analysis Platform of Resources and Environment
DOI | 10.1029/2019WR024906 |
A Feature-Based Procedure for Detecting Technical Outliers in Water-Quality Data From In Situ Sensors | |
Talagala, Priyanga Dilini1,2; Hyndman, Rob J.1,2; Leigh, Catherine1,3,4; Mengersen, Kerrie1,4; Smith-Miles, Kate1,5 | |
2019-11-06 | |
发表期刊 | WATER RESOURCES RESEARCH
![]() |
ISSN | 0043-1397 |
EISSN | 1944-7973 |
出版年 | 2019 |
文章类型 | Article;Early Access |
语种 | 英语 |
国家 | Australia |
英文摘要 | Outliers due to technical errors in water-quality data from in situ sensors can reduce data quality and have a direct impact on inference drawn from subsequent data analysis. However, outlier detection through manual monitoring is infeasible given the volume and velocity of data the sensors produce. Here we introduce an automated procedure, named oddwater, that provides early detection of outliers in water-quality data from in situ sensors caused by technical issues. Our oddwater procedure is used to first identify the data features that differentiate outlying instances from typical behaviors. Then, statistical transformations are applied to make the outlying instances stand out in a transformed data space. Unsupervised outlier scoring techniques are applied to the transformed data space, and an approach based on extreme value theory is used to calculate a threshold for each potential outlier. Using two data sets obtained from in situ sensors in rivers flowing into the Great Barrier Reef lagoon, Australia, we show that oddwater successfully identifies outliers involving abrupt changes in turbidity, conductivity, and river level, including sudden spikes, sudden isolated drops, and level shifts, while maintaining very low false detection rates. We have implemented this oddwater procedure in the open source R package oddwater. |
领域 | 资源环境 |
收录类别 | SCI-E |
WOS记录号 | WOS:000494575600001 |
WOS关键词 | NETWORKS |
WOS类目 | Environmental Sciences ; Limnology ; Water Resources |
WOS研究方向 | Environmental Sciences & Ecology ; Marine & Freshwater Biology ; Water Resources |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://119.78.100.173/C666/handle/2XK7JSWQ/188269 |
专题 | 资源环境科学 |
作者单位 | 1.ARC Ctr Excellence Math & Stat Frontiers ACEMS, Melbourne, Vic, Australia; 2.Monash Univ, Dept Econmetr & Business Stat, Clayton, Vic, Australia; 3.Queensland Univ Technol, Inst Future Environm, Sci & Engn Fac, Brisbane, Qld, Australia; 4.Queensland Univ Technol, Sch Math Sci, Sci & Engn Fac, Brisbane, Qld, Australia; 5.Univ Melbourne, Sch Math & Stat, Parkville, Vic, Australia |
推荐引用方式 GB/T 7714 | Talagala, Priyanga Dilini,Hyndman, Rob J.,Leigh, Catherine,et al. A Feature-Based Procedure for Detecting Technical Outliers in Water-Quality Data From In Situ Sensors[J]. WATER RESOURCES RESEARCH,2019. |
APA | Talagala, Priyanga Dilini,Hyndman, Rob J.,Leigh, Catherine,Mengersen, Kerrie,&Smith-Miles, Kate.(2019).A Feature-Based Procedure for Detecting Technical Outliers in Water-Quality Data From In Situ Sensors.WATER RESOURCES RESEARCH. |
MLA | Talagala, Priyanga Dilini,et al."A Feature-Based Procedure for Detecting Technical Outliers in Water-Quality Data From In Situ Sensors".WATER RESOURCES RESEARCH (2019). |
条目包含的文件 | 条目无相关文件。 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论