AzureException: Unable to access container using anonymous credentials, and no credentials found for them in the configuration

Question

I am trying to use Hadoop of Azure HDInsight. I am logging into the cluster by ssh and running the following

hadoop jar jar_name class_name wasb://[email protected]/inputdir wasb://[email protected]/outputdir

But I get the following exception:

Exception in thread "main" org.apache.hadoop.fs.azure.AzureException: org.apache.hadoop.fs.azure.AzureException: Unable to access container xxx in account yyy.core.windows.net using anonymous credentials, and no credentials found for them in the configuration.

I am using azure cli and I ran "azure login" before running the above mentioned hadoop job.

Please let me know how I can fix this error.

Ubuntu 12.04 runs on these machines.

PS: posted this on Azure forums as well, but reposting it here to get broader audience.

Andrew Moll · Accepted Answer · 2015-11-12T21:09:38.513

12

EDIT: Your storage location should be wasb://[email protected]/inputdir

It sounds like your cluster is not attached to that storage account. Recreate your cluster and make sure that the account that holds the jar is attached to the cluster. Otherwise, move your jar to a storage account that is connected!

edited Nov 12 '15 at 21:09

answered Nov 10 '15 at 22:28

Andrew Moll

4,903
2
13
15

I am using the same storage but a different container from the one I specified during cluster creation time. I also created a public container within the same storage. I am unable to access this container as well using a command such as hdfs dfs -ls wasb://[email protected] where data2 is a public container. – Raghava Nov 10 '15 at 23:22
1

Try hdfs dfs -ls wasb://[email protected] – Andrew Moll Nov 11 '15 at 16:38
Sorry* hdfs dfs -ls wasb://[email protected]/inputdir You should always have your files in an input directory. – Andrew Moll Nov 12 '15 at 21:00
Problem is with the missing trailing '/'. There should be a / at the end. – Raghava Nov 12 '15 at 21:05

score 3 · Answer 2 · answered Jan 06 '21 at 18:21

3

The correct format to use Azure Blob storage is

wasbs://<container>@<storage-account-name>.blob.core.windows.net/<directory>/

Always make sure to end your wasbs with /

answered Jan 06 '21 at 18:21

Amit Shahi

81
5

score 2 · Answer 3 · answered Nov 12 '15 at 21:07

2

This problem has been resolved and I am putting up the answer here in case someone faces the same issue.

I made couple of silly mistakes.

1) It should be wasb://@.blob.core.windows.net. I missed 'blob' earlier. 2) There should be a trailing '/'. So it should be hdfs dfs -ls wasb://[email protected]/

answered Nov 12 '15 at 21:07

Raghava

947
4
15
29

1

and you missed blob again in the last link – Yauheni Leaniuk May 30 '22 at 14:29

score 0 · Answer 4 · answered Mar 26 '20 at 18:43

0

I faced this error.

I used to write

wasb://directory/

instead of

wasb://[email protected]/directory/

answered Mar 26 '20 at 18:43

Ganesh

677
8
11

AzureException: Unable to access container using anonymous credentials, and no credentials found for them in the configuration

4 Answers4

Linked