Exploring_Mapreduce/TV Sales.txt at master · Dunc68/Exploring_Mapreduce · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
Copy file to HDFS

[acadgild@localhost Documents]$ hadoop fs -copyFromLocal TVSales.jar TVSales.jar
17/09/03 14:16:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Run Job

[acadgild@localhost Documents]$ yarn jar TVSales.jar  television.txt tv_sales
17/09/03 14:16:54 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/09/03 14:16:55 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/09/03 14:16:56 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/09/03 14:16:56 INFO input.FileInputFormat: Total input paths to process : 1
17/09/03 14:16:56 INFO mapreduce.JobSubmitter: number of splits:1
17/09/03 14:16:56 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1504029604882_0017
17/09/03 14:16:57 INFO impl.YarnClientImpl: Submitted application application_1504029604882_0017
17/09/03 14:16:57 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1504029604882_0017/
17/09/03 14:16:57 INFO mapreduce.Job: Running job: job_1504029604882_0017
17/09/03 14:17:03 INFO mapreduce.Job: Job job_1504029604882_0017 running in uber mode : false
17/09/03 14:17:03 INFO mapreduce.Job:  map 0% reduce 0%
17/09/03 14:17:08 INFO mapreduce.Job:  map 100% reduce 0%
17/09/03 14:17:14 INFO mapreduce.Job:  map 100% reduce 100%
17/09/03 14:17:14 INFO mapreduce.Job: Job job_1504029604882_0017 completed successfully
17/09/03 14:17:14 INFO mapreduce.Job: Counters: 49
        File System Counters
                FILE: Number of bytes read=204
                FILE: Number of bytes written=212533
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=848
                HDFS: Number of bytes written=38
                HDFS: Number of read operations=6
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=2
        Job Counters
                Launched map tasks=1
                Launched reduce tasks=1
                Data-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=3067
                Total time spent by all reduces in occupied slots (ms)=3181
                Total time spent by all map tasks (ms)=3067
                Total time spent by all reduce tasks (ms)=3181
                Total vcore-seconds taken by all map tasks=3067
                Total vcore-seconds taken by all reduce tasks=3181
                Total megabyte-seconds taken by all map tasks=3140608
                Total megabyte-seconds taken by all reduce tasks=3257344
        Map-Reduce Framework
                Map input records=18
                Map output records=16
                Map output bytes=166
                Map output materialized bytes=204
                Input split bytes=115
                Combine input records=0
                Combine output records=0
                Reduce input groups=5
                Reduce shuffle bytes=204
                Reduce input records=16
                Reduce output records=5
                Spilled Records=32
                Shuffled Maps =1
                Failed Shuffles=0
                Merged Map outputs=1
                GC time elapsed (ms)=124
                CPU time spent (ms)=1400
                Physical memory (bytes) snapshot=417820672
                Virtual memory (bytes) snapshot=4161044480
                Total committed heap usage (bytes)=343932928
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=733
        File Output Format Counters
                Bytes Written=38


[acadgild@localhost Documents]$ hadoop fs -ls tv_sales
17/09/03 14:18:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-r--r--   1 acadgild supergroup          0 2017-09-03 14:17 tv_sales/_SUCCESS
-rw-r--r--   1 acadgild supergroup         38 2017-09-03 14:17 tv_sales/part-r-00000
[acadgild@localhost Documents]$ hadoop fs -cat tv_sales/part-r-00000
17/09/03 14:18:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Akai    1
Lava    3
Onida   3
Samsung 7
Zen     2