All posts by Alex Volok
Azure Data Factory and REST APIs – Mapping and Pagination
In this blog post I would like to put a light on a mapping and pagination of a Copy activity which are often are requirements for the ingestion of REST data.Mapping is optional for data sources, like relational databases, parquet or csv files, because the incoming dataset contains an inherited structure, so the Data Factory […]
Azure Data Factory and REST APIs – Setting up a Copy activity
In this blog post, I would like to show how to add and configure a Copy Activity which will ingest REST data and store it in a data lake. The REST API used as an example in this post is secured with OAUTH2 authentication. It requires a Bearer token to be obtained and passed as […]
Azure Data Factory and REST APIs – Dealing with oauth2 authentication
In this first post I am going to discuss how to apply oauth2 authentication to ingest REST APIs data. OAUTH2 became a standard de facto in cloud and SaaS services, it used widely by Twitter, Microsoft Azure, Amazon. My example is based on EXACT Online API. The end-target of the blog series is to setup […]
Azure Data Lake Gen2 and SQL Server Integration Services
How it used to be “Azure Data Lake Generation 2” (or ADLS gen2) is the newest cloud data lake offering from a Microsoft. It supposes to bring the best of two worlds together: excelent performance and redundancy of a blob storage and secure filesystem capabilities of a data lake. However, for some time the ADLS […]
Automated Refresh of Databases on Development Servers
Databases that have to be copied from a production server It happens that databases in testing and especially acceptance environments needed to be refreshed by restoring backups from a production environment. The backup restore is seems like a trivial operation that can be done via SQL Server Management Studio wizard or via prepared T-SQL scripts.However, […]