Tuesday, August 8, 2017

Storing Family data in the cloud


We are creating more and more data with the arrival of the digital camera and cell phone camera.  But what about all the pictures you have already. How do you save those photos and at the same time share them with your family?  You could use Facebook as place to share old family photos and gather information but Facebook isn't designed for long term storage, data retrieval and archiving of high quality photos.

 

Enter Azure and Blob Storage (https://azure.microsoft.com/en-us/services/storage/blobs/)

 

Put simply a Blob is unstructured data you store for retrieval via HTTP or HTTPS.  Data such as text files, pictures, or streaming files like Audio or Video. This data can be easily retrievable or stored for the long term.

 

Cool Storage - Files that you don't expect to access very often like a long term backup or possible the high resolution version of a Family Picture.

Hot Storage -File that you expect to use often and need readily available like a thumbnail or small 100k version of your high-resolution picture.

 

Understand that to make a hot file cool is free but if you want to thaw out your data for retrieval, there is a cost.

 

Now that we understand what a Blob is, how do we organize our data. 

 

Files are stored in Containers (https://docs.microsoft.com/en-us/azure/storage/storage-dotnet-how-to-use-blobs) like Folders in your My Documents.  Pictures, Videos or you could break it down such as Smith Family, Jones Family if you trying to save your videos from a family tree.

 

Now that we have talked about the basics.  Let's look at what we can do with Azure and why data storage retrieval can be easy and benificial to many people distributed around the world outside of the business area.

 

Problem: Person has tons of family photos but doesn't know the identities of the people.  Several family members have helped out but members are all over the globe and it would be nice to catalog the data so you can search as well as download a high-resolution version of a family picture. 

 

Solution: Azure Blob Storage, Azure SQL and possibly a Web Front end all in the Central Region

 

I will take a simplified view of the cost of what it might take to get it started:

In the future, we will walk through the process of setting it up, right now we talk about the cost of getting started.

 

Blob Storage Cost

https://azure.microsoft.com/en-us/pricing/details/storage/blobs/

Cool Storage - Geographically Redundant Storage (GRS) because we want our data separated and backed up

Hot Storage - Locally Redundant Storage (LRS), these are the thumbnails and preview files that are small and won't matter if lost.

(https://docs.microsoft.com/en-us/azure/storage/storage-blob-storage-tiers)

 

Recall - Your cold storage is the high-resolution files that you shouldn't be viewing often.

 

10 family containers

5 hot - .05

5 cool - .20

 

Retrieving the data

5 hot - free

5 cool - $0.01/GB

 

There is a cost to store and replicate your data for later use

Saving and Replicating the data

Hot - Free

Cool - $0.02

 

 Azure SQL is required for us to save the metadata for the files like location, who is in the pictures, etc

 

https://azure.microsoft.com/en-us/pricing/details/sql-database/

 

Databases under 2GB is almost free at $0.0068/hr of access.  Given this data is only access when you search or write and this should be relatively small.  Should it grow beyond 2GB then you start to see your cost increase.

 

Ideally the database will show a picture no more than 100kb. Videos and Audio will need to be compress to smaller file so you don't pay for previewing the data.

 

But how do we access the data?  You create your Azure database here (https://docs.microsoft.com/en-us/azure/sql-database/) add a specific IP address of your local machine and then you can create an MS Access front end to access your data.  Realize that searching and moving through will incur a cost. But you can also have a button to download the Cool Storage of the file.  When you perform this action, Cool Storage will be converted to Hot storage.

 

"Changing the storage tier: Changing the storage tier from cool to hot will incur a charge equal to reading all the data existing in the storage account for every transition. On the other hand, changing the storage tier from hot to cool will be free of cost."