Checking for date validity in spark sql -

is possible check if date(yyymmdd) valid in spark sql? have tried following:

to_date(cast(unix_timestamp(cast(unix_timestamp(colname, 'yyyymmdd')  timestamp))) not null 

thanks, bharath

you need use dater parser similar natty (depending on language using) natty.joestelmach.com

you can create function parses , validates date using natty (or other parser). register udf can use in spark sql.

val isvaliddate: string => boolean =//your validation logic here sqlcontext.registerfunction("datevalidate", isvaliddate) sqlcontext.sql("select * df datevalidate(colname)")