Most Popular
1500 questions
40
votes
4 answers
What are the advantages of HDF compared to alternative formats?
What are the advantages of HDF compared to alternative formats? What are the main data science tasks where HDF is really suitable and useful?
IgorS
- 5,474
- 11
- 31
- 43
40
votes
10 answers
Do I need to learn Hadoop to be a Data Scientist?
An aspiring data scientist here. I don't know anything about Hadoop, but as I have been reading about Data Science and Big Data, I see a lot of talk about Hadoop. Is it absolutely necessary to learn Hadoop to be a Data Scientist?
Pensu
- 591
- 1
- 4
- 8
40
votes
4 answers
Why do we need XGBoost and Random Forest?
I wasn't clear on couple of concepts:
XGBoost converts weak learners to strong learners. What's the advantage of doing this ? Combining many weak learners instead of just using a single tree ?
Random Forest uses various sample from tree to create…
John Constantine
- 697
- 2
- 8
- 10
40
votes
3 answers
Why use both validation set and test set?
Consider a neural network:
For a given set of data, we divide it into training, validation and test set. Suppose we do it in the classic 60:20:20 ratio, then we prevent overfitting by validating the network by checking it on validation set. Then…
user1825567
- 1,396
- 1
- 12
- 22
40
votes
8 answers
Using TensorFlow with Intel GPU
Is there any way now to use TensorFlow with Intel GPUs? If yes, please point me in the right direction.
If not, please let me know which framework, if any, (Keras, Theano, etc) can I use for my Intel Corporation Xeon E3-1200 v3/4th Gen Core…
James Bond
- 1,195
- 2
- 11
- 12
40
votes
1 answer
The difference between `Dense` and `TimeDistributedDense` of `Keras`
I am still confused about the difference between Dense and TimeDistributedDense of Keras even though there are already some similar questions asked here and here. People are discussing a lot but no common-agreed conclusions.
And even though, here,…
fluency03
- 503
- 1
- 5
- 8
39
votes
7 answers
How to get sentence embedding using BERT?
How to get sentence embedding using BERT?
from transformers import BertTokenizer
tokenizer=BertTokenizer.from_pretrained('bert-base-uncased')
sentence='I really enjoyed this movie a lot.'
#1.Tokenize the…
star
- 1,471
- 7
- 19
- 29
39
votes
8 answers
In a Transformer model, why does one sum positional encoding to the embedding rather than concatenate it?
While reviewing the Transformer architecture, I realized something I didn't expect, which is that :
the positional encoding is summed to the word embeddings
rather than concatenated to…
FremyCompany
- 503
- 1
- 4
- 7
39
votes
4 answers
Applications and differences for Jaccard similarity and Cosine Similarity
Jaccard similarity and cosine similarity are two very common measurements while comparing item similarities. However, I am not very clear in what situation which one should be preferable than another.
Can somebody help clarify the differences of…
shihpeng
- 553
- 1
- 4
- 8
39
votes
2 answers
Keras difference beetween val_loss and loss during training
What is the difference between val_loss and loss during training in Keras?
E.g.
Epoch 1/20
1000/1000 [==============================] - 1s - loss: 0.1760, val_loss: 0.2032
On some sites I read that on validation, dropout was not working.
Vladimir Shebuniayeu
- 559
- 1
- 4
- 9
39
votes
1 answer
How does Keras calculate accuracy?
How does Keras calculate accuracy from the classwise probabilities? Say, for example we have 100 samples in the test set which can belong to one of two classes. We also have a list of the classwise probabilites. What threshold does Keras use to…
pseudomonas
- 1,042
- 3
- 14
- 30
38
votes
5 answers
Are decision tree algorithms linear or nonlinear
Recently a friend of mine was asked whether decision tree algorithms are linear or nonlinear algorithms in an interview. I tried to look for answers to this question but couldn't find any satisfactory explanation. Can anyone answer and explain the…
user2966197
- 521
- 1
- 6
- 8
38
votes
5 answers
Best practices to store Python machine learning models
What are the best practices to save, store, and share machine learning models?
In Python, we generally store the binary representation of the model, using pickle or joblib. Models, in my case, can be ~100Mo large. Also, joblib can save one model to…
Antoine Dusséaux
- 481
- 1
- 4
- 7
38
votes
4 answers
What is the meaning of "The number of units in the LSTM cell"?
From Tensorflow code: Tensorflow. RnnCell.
num_units: int, The number of units in the LSTM cell.
I can't understand what this means. What are the units of LSTM cell? Input, Output and Forget gates? Does this mean "the number of units in the…
Brans Ds
- 849
- 1
- 8
- 18
38
votes
3 answers
Calculation and Visualization of Correlation Matrix with Pandas
I have a pandas data frame with several entries, and I want to calculate the correlation between the income of some type of stores. There are a number of stores with income data, classification of area of activity (theater, cloth stores, food ...)…
gdlm
- 535
- 1
- 6
- 9