57 lines
938 B
Markdown
57 lines
938 B
Markdown
# Pruebas Cluster spark
|
|
|
|
## Comandos:
|
|
|
|
### Instalar Java
|
|
|
|
```
|
|
sudo apt install -y openjdk-8-jdk
|
|
|
|
```
|
|
|
|
### Iniciar master
|
|
|
|
```
|
|
ssh cluster_master ~/Documentos/spark/sbin/start-master.sh
|
|
```
|
|
### Detener master
|
|
|
|
```
|
|
ssh cluster_master ~/Documentos/spark/sbin/stop-master.sh
|
|
|
|
```
|
|
|
|
|
|
### Iniciar Workers
|
|
|
|
```
|
|
ssh cluster_master ~/Documentos/spark/sbin/start-workers.sh
|
|
```
|
|
### Detener Workers
|
|
|
|
```
|
|
ssh cluster_master ~/Documentos/spark/sbin/stop-workers.sh
|
|
|
|
```
|
|
|
|
|
|
## Carga de archivos
|
|
|
|
Para cargar datos es necesario enviarlos a todos los nodos , se podria enviar a cada nodo con **scp** o crear un archivo con el nombre del host de cada nodo.
|
|
|
|
```bash
|
|
for i in `cat slaves`; do
|
|
scp data.csv $i:~/dataSpark/
|
|
done
|
|
```
|
|
|
|
### Crear carpeta de datos
|
|
Todo:
|
|
|
|
- [ ] montar carpeta compartida Samba
|
|
|
|
```bash
|
|
for i in `cat slaves`; do
|
|
ssh $i mkdir ~/dataSpark
|
|
done
|
|
``` |